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PREFACE 


This  report  is  addressed  to  the  technical  and  managerial  personnel 
in  the  aerospace  field  who  have  a  part  in  the  planning  and  development 
of  modern  aircraft.  It  is  particularly  directed  to  those  executives  within 
the  Air  Force  and  Department  of  Defense  who  have  within  their  power  the 
controls  to  make  changes  in  the  aircraft  development  and  acquisition 
cycle. 

The  investigation  described  in  this  report  has  been  concentrated  on 
the  problems  associated  with  environmental  test  facilities  and  their  role 
in  the  development  of  aerospace  systems.  The  environmental  test  facility 
is  defined  as  ground-based  (as  opposed  to  airborne)  test  equipment  which 
either  duplicates  or  simulates  natural  and  induced  environmental  parameters 
which  are  important  to  the  test  article.  The  test  article  may  be  a  full-size 
flight  vehicle  or  component,  or  a  scaled-down  version  of  the  same.  Hope¬ 
fully,  the  measurements  made  on  the  test  article  in  the  test  facility  will 
permit  prediction  of  vehicle  or  component  performance  which  closely 
approximates  that  to  be  achieved  in  the  real  flight  environment.  Wind 
tunnels,  turbine  and  rocket  engine  test  cells,  vacuum  chambers,  and  a 
host  of  other  test  devices  are  included  in  this  category.  Although  much 
of  the  philosophy  expressed  herein  is  applicable  to  all  pre-flight  aircraft 
evaluation  facilities,  this  study  is  directed  primarily  toward  the  use  and 
usefulness  of  wind  tunnels  in  the  aircraft  development  cycle.  Except 
where  defined  otherwise,  "testing"  will  mean  wind  tunnel  programs. 

The  author  performed  this  study  for  the  Air  Force  Systems  Command 
while  on  sabbatical  at  Vanderbilt  University.  However,  his  past  exper¬ 
iences  at  the  Arnold  Engineering  Development  Center  in  the  operation, 
planning  and  design  of  aerospace  test  facilities  were  most  instrumental 
in  the  construction  of  the  investigation.  It  has  been  the  author's  obser- 
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vation  that  many  of  the  problems  in  aerospace  systems  development  have 
been  the  result  of  Improper  use  of  and  inadequacies  in  testing  facilities 
and  testing  techniques.  Some  of  the  more  serious  problems  are  not  sub¬ 
ject  to  technical  solutions,  for  they  are  the  result  of  managerial  practices 
and  established  procedures.  Sometimes  the  cause  of  the  problem  is  diffi¬ 
cult  to  define,  for  it  may  be  the  result  of  a  seemingly  attractive  manage¬ 
ment  position  in  a  related  area. 

The  arguments  and  conclusions  presented  in  the  following  chapters 
are  not  solely  those  of  the  author;  instead,  hypotheses  and  problems 
posed  by  the  author  have  been  evaluated  by  many  of  the  experienced  and 
knowledgeable  experts  who  are  Instrumental  in  the  development  of  today's 
aircraft.  A  large  portion  of  the  aircraft  development  projects  of  the  past 
two  decades  have  been  studied  in  an  attempt  to  ascertain  the  influence  of 
the  test  facility  program  upon  development  successes  and  failures.  The 
view  is  not  that  of  the  historian;  the  evaluations  are  by  those  who  helped 
make  the  history.  Such  studies  over  many  aircraft  development  projects 
have  permitted  some  meaningful  tests  of  hypotheses  and  the  generation  of 
a  theory  to  optimize  the  use  of  test  facilities  in  the  aircraft  development 
cycle. 

It  is  not  the  purpose  of  this  report  to  criticize  individuals  or  organi¬ 
zations  for  their  past  decisions  or  actions.  Too  often  an  evaluation  of 
historical  data  is  so  interpreted.  It  would  be  impossible  to  reconstruct 
the  many  extenuating  circumstances  which  prompted  some  of  the  past 
decisions.  Yet,  the  consequences  of  some  of  these  actions  can  be 
evaluated;  further,  some  of  the  situations  which  forced  individuals  into 
unfortunate  and  compromising  situations  can  be  analyzed  in  retrospect. 

The  reader  is  asked  to  examine  the  arguments  and  analyses  in  the  follow¬ 
ing  chapters  without  trying  to  defend  any  previous  decisions  and  activities. 
Instead,  he  is  asked  to  review  the  effects  of  such  actions  as  assessed  by 
a  number  of  distinguished  experts  so  that  he  may  improve  his  position  for 
making  future  decisions. 
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A  study  of  this  type  would  not  be  possible  without  the  help  of  many 
people.  The  author  is  indebted  to  numerous  aerospace  experts  in  the 
government,  industry,  and  universities  who  so  generously  responded  to 
the  questionnaires  and  interviews;  a  list  of  these  individuals  is  included 
as  Appendix  A  of  this  report.  Though  not  identified,  the  secondary  con¬ 
tributors  and  coordinators  played  an  important  role  in  the  collection  of  the 
data.  The  interest  that  so  prestigious  a  group  has  shown  through  volun¬ 
tary  and  detailed  responses  and  the  desire  of  so  many  to  be  apprised  of 
the  results  of  this  study  have  been  most  encouraging. 

A  special  debt  of  gratitude  is  reserved  for  individuals  at  Headquarters, 
Air  Force  Systems  Command,  and  Headquarters,  U.  S.  Air  Force,  who  have 
provided  the  funds  necessary  to  gather  the  data  and  have  supported  the 
author  in  many  other  ways.  Particular  mention  should  be  made  of  Brig. 

Gen.  John  D.  Peters,  Maj.  Harold  Nelson,  Jr.,  and  Mr.  Jerome  S.  Kamchi. 
The  author  also  wishes  to  acknowledge  the  help  of  his  friends  and  asso¬ 
ciates  in  the  Requirements  Planning  Division,  Directorate  Plans  and  Tech¬ 
nology,  Headquarters,  Arnold  Engineering  Development  Center,  for  their 
assistance  in  searching  out  some  of  this  information. 

Finally,  the  author  wishes  to  express  his  deep  appreciation  to  the 
following  members  of  the  Vanderbilt  University  Faculty  who  have  provided 
advice  and  consultation  during  this  study;  Dr.  Edwin  M.  Bartee,  Prof. 
Bruce  M.  Bayer,  Dr.  Charles  E.  Goshen,  Dr.  John  W.  Williamson,  and 
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EXECUTIVE  SUMMARY 


The  purpose  of  this  summary  is  to  provide  a  quick  review  of  the 
six-chapter  report  on  the  use  and  usefulness  of  wind  tunnels  and 
related  environmental  test  facilities  in  the  aircraft  development  process. 
The  author  has  tested  hypotheses  and  explored  problems  by  way  of  ques¬ 
tionnaires  and  interviews;  comments  and  judgements  of  117  of  the 
nation's  leading  aerospace  experts  are  recorded  in  the  report.  This  sum¬ 
mary  provides  the  major  findings  of  the  study,  and  covers  the  author's 
recommendations  for  corrective  action  and  further  Investigation 

The  evaluation  of  the  origin  of  the  facility  test  plan*^  provided  the 
following  conclusions  and  observations: 

1.  A  reduction  in  the  magnitude  or  quality  of  the  facility  test  program 
is  associated  with  an  increase  in  technical  risk  in  aircraft  development. 

2.  The  competitive  nature  of  the  bidding  procedure  for  aircraft  devel¬ 
opment  contracts  handicaps  the  origination  and  acceptance  of  a  good 
facility  test  plan.  The  system  acts  to  force  test  plans  toward  inadequacy 
because: 

a.  The  usual  emphasis  on  low  development  cost  and  short 
development  time  encourages  a  potential  contractor  to 
minimize  his  test  plan. 

b.  There  is  a  tendency  to  reduce  attention  to  the  test  plan 
because  of  the  many  other  items  which  may  be  considered 
more  important  in  the  origination  and  evaluation  of  the 
bid  proposal. 

^An  asterisk  indicates  that  the  phrase  or  word  is  further  defined  or 
explained  in  the  Glossary. 
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c.  The  test  plan  is  relatively  unimportant  in  the  evaluation 
of  the  bid  or  in  selection  of  a  contractor. 

d.  The  potential  contractor  is  sometimes  hesitant  to  include 
a  substantial  test  program  in  a  given  technical  area  be¬ 
cause  it  may  be  interpreted  as  an  indication  of  uncer¬ 
tainty  on  his  part  and  work  against  his  acceptance, 

3.  The  test  plan  is  primarily  developed  by  the  contractor,  with  some 
guidance  by  the  developing  organization.  The  Arnold  Engineering  Develop¬ 
ment  Center  (AEDC)*  facility  personnel  have  very  little  direct  or  indirect 
influence  on  the  test  plan.  The  National  Aeronautics  and  Space  Administra¬ 
tion  (NASA)*  test  personnel  exert  more  Influence  than  those  at  AEDC  on  the 
facility  test  plans  for  development  of  aircraft. 

4.  There  is  overwhelming  agreement  among  respondents  from  partic¬ 
ipating  agencies  that  some  changes  should  be  made  in  the  method  of  pre¬ 
paring  and  evaluating  the  test  plan.  Many  respondents  recommend  that 
more  importance  be  attached  to  the  test  plan  in  the  evaluation  of  a  bid; 
others  recommend  that  the  test  program  be  originated  in  closer  coordina¬ 
tion  with  ASD  and  AEDC  and  added  to  the  contract  on  a  cost-plus-fixed- 
fee  basis. 

5.  There  are  indications  that  service  funding*  at  AEDC  is  causing 
unfavorable  working  relations  between  AEDC  and  the  Aeronautical  Systems 
Division  (ASD).  *  This  is  brought  about  by  AEDC's  incentive  to  direct  test 
workload  to  its  facilities  and  ASD's  incentive  to  test  in  "free"  or  cheaper 
facilities.  A  result  appears  to  be  less  invited  participation  for  AEDC  in 
establishing  or  advising  on  facility  test  programs. 

The  study  of  the  test  facility's  role  in  implementation  of  the  new 
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Department  of  Defense  (DoD)  *  development  philosophy  provided  the 
following  conclusions  and  observations: 


The  new  DoD  development  philosophy  is  discussed  in  Chapters  I 

and  IV. 
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1.  There  is  strong  agreement  among  respondents  with  that  portion  of 
the  new  DoD  systems  development  philosophy  which  emphasizes  demon¬ 
stration  of  performance  and  reduction  of  risk  early  in  the  development 
program.  The  test  facility  is  seen  to  have  a  major  role  in  implementation 
of  the  policy  and  more  wind  tunnel  testing  may  be  reguired  in  the  develop¬ 
ment  cycle. 

2.  Many  respondents  do  not  believe  that  the  government  will  change 
its  method  of  operation  and  provide  sufficient  time  and  funds  early  in  the 
development  cycle  to  permit  implementation  of  the  new  policy.  Already 
facility  testing  is  being  de-emphasized  because  of  greater  emphasis  on 
demonstration  of  performance  in  flight  prior  to  a  production  decision.  The 
industry  designer  does  not  believe  that  the  government  decision-maker  is 
often  aware  of  his  dependence  upon  data  from  the  test  facility  and  his 
ability  to  assess  technical  risk  and  provide  a  good  design  based  on  this 
data. 

3.  The  test  facility  can  provide  valuable  and  tangible  information 
to  the  government  decision-maker  for  his  Program  Decision  and  Ratifica- 
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tion  Decision.  Milestone  checks  of  hardware  performance  and  risk 
identification  in  test  facilities  are  seen  as  a  most  important  aspect  of 
aircraft  systems  development. 

4.  Competitive  full-scale  flight  prototypes*  are  not  expected  to  be 
a  part  of  the  normal  development  cycle  because  of  the  costs  involved; 
this  is  especially  true  for  the  large  or  technically  advanced  aircraft. 

Even  for  the  less  sophisticated  and  inexpensive  aircraft,  there  is  con¬ 
cern  that  an  attempt  to  fund  dual  development  programs  with  limited  funds 
will  result  in  short  development  schedules  and  little  use  of  test  facilities 

4 

(e.  g.  ,  the  A-X) .  The  advantages  of  the  added  competition  are  not 


Program  Decision  and  Ratification  Decision  are  discussed  in  Chap¬ 
ter  IV. 
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The  A-X  specialized  close-air- support  aircraft  is  presently  under 
development  for  the  Air  Force. 
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expected  to  compensate  for  lack  of  design  data. 

5.  Competitive  "model  fly-offs"*  in  the  government  facility  during 

the  Validation  Phase^  is  an  acceptable  compromise  between  "paper  studies"* 
and  full-scale  fly-offs*  for  many  respondents.  This  approach  provides  a 
competitive  atmosphere  for  development,  gives  the  government  evaluator 
unbiased  and  hard  data  upon  which  to  base  his  judgements,  yet  does  not 
incur  the  expense  of  flight  prototypes. 

6.  The  interpretation  and  evaluation  of  validation  information  taken 
from  the  test  facility  can  have  an  influence  upon  substantiation  of  mile¬ 
stones  and  major  program  decisions.  The  existing  procedures  within  the 
Air  Force  Systems  Command  (AFSC)*  do  not  always  assure  the  processing 
of  the  information  with  the  requisite  objectivity  and  freedom  from  biases. 

This  situation  may  be  expected  when  the  evaluation  and  development  func¬ 
tions  are  not  exercised  independently. 

7.  The  governtnent  should  determine  compliance  of  the  contractor's 
product  in  government  facilities.  These  facilities  should  be  the  best 
available  to  reduce  misinterpretation  of  the  data. 

8.  Test  personnel  at  government  facilities  should  have  the  capacity 
to  advise  the  user  on  the  accuracy  of  the  data  that  comes  from  the  facility 
and  give  a  good  estimate  of  the  correlation  to  be  expected  with  flight  data. 

The  investigation  of  test  facility  deficiencies  and  their  consequences 
resulted  in  the  following  conclusions  and  observations; 

1.  The  deficiencies  in  environmental  simulation  in  aeronautical 
test  facilities  have  had  detrimental  effects  upon  the  development  of  aero¬ 
space  systems.  The  major  deficiencies  most  often  mentioned  by  the 
respondents  were: 

a.  Lack  of  Reynolds  number  capability  at  transonic  speeds. 

^The  Validation  Phase  of  systems  development  is  discussed  in 
Chapter  IV. 
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b.  Lack  of  a  facility  to  investigate  the  areas  of  inlet-engine 
compatibility  and  nozzle-airframe  integration. 

c.  Lack  of  a  facility  to  support  development  of  larger  and 
more  powerful  subsonic  and  supersonic  turbofan  and 
turbojet  engines. 

d.  Almost  complete  lack  of  good  hypersonic  development 
facilities. 

2.  More  emphasis  on  testing  techniques  and  data  Interpretation  in 
existing  facilities  can  substantially  improve  the  contribution  of  these 
facilities  to  aircraft  development. 

3.  Though  not  prevalent,  the  deficiencies  in  facility  test  capability 
are  sometimes  causing  compromises  in  aircraft  design  and  performance 
because  the  designer  or  planner  is  aware  of  the  limits  of  the  available 
facilities;  he  does  not  choose  to  propose  hardware  or  advanced  technical 
ideas  which  would  either  necessitate  new  test  facilities  or  be  developed 
with  high  risk. 

4.  Most  respondents  expect  test  facility  deficiencies  to  have  an 
even  greater  impact  upon  aircraft  development  during  the  next  decade. 

The  results  of  the  deficiencies  are  anticipated  to  be  stifled  technology, 
more  costly  development  programs,  higher  risk  developments,  and  less 
than  desired  performance  in  future  aircraft. 

5.  It  is  estimated  that  about  15%  of  the  modern  facility  test  program 
is  a  result  of  similar  tests  performed  in  different  facilities  in  an  attempt 
to  verify  data;  this  is  because  of  uncertainty  in  the  extrapolation  of 
simulation  parameters.  The  primary  cause  for  duplicate  testing  is  inade¬ 
quate  Reynolds  number. 

The  following  remarks  resulted  from  a  study  of  the  role  of  contractor 
test  facilities  and  government  test  facilities: 

1.  Most  respondents  believe  that  the  aerospace  contractor  should 
have  certain  low-cost  backyard  test  facilities.  *  Industry  respondents 
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say  that  such  facilities  provide  quick  response  and  flexibility  to  the  con¬ 
tractor,  and  are  the  key  to  industry  research  and  contractor  funded  early 
development  prior  to  government  involvement.  Although  most  government 
respondents  acknowledge  the  usefulness  of  industry  backyard  facilities, 
many  feel  that  "contractor  owned"  facilities  are  essentially  paid  for  by 
the  government  as  the  charges  and  overhead  are  prorated  over  a  number  of 
aircraft  development  projects;  they  argue  that  the  government  should  have 
better  control  over  the  number  of  such  facilities  developed  indirectly  with 
DoD  funds. 

2.  Most  respondents  agree  that  the  government  should  provide  the 
high-cost  test  facilities  as  "national"  test  facilities  for  the  use  of  every¬ 
one.  However,  a  few  respondents  would  prefer  to  see  such  facilities 
built  by  industry  on  a  cooperative  basis. 

3.  Both  industry  and  government  respondents  overwhelmingly  agree 
that  contractor-owned  aeronautical  test  facilities  assist  the  company  in 
obtaining  government  contracts.  This  is  true  whether  or  not  the  same 
test  capability  is  available  for  the  contractor's  use  at  government  facili¬ 
ties.  Thus,  a  competitive  situation  has  been  created  whereby  the  ex¬ 
cellence  of  a  company's  test  facilities  has  a  strong  influence  upon  its 
ability  to  capture  new  development  contracts;  this  situation  has  resulted 
in  proliferation  of  test  facilities  by  industry. 

4.  Regulations  and  directives  which  force  testing  at  certain  AFSC 
test  facilities  are  the  subject  of  much  concern  among  aerospace  personnel. 
There  are  strong  opinions  on  both  sides  of  the  argument,  and  the  situation 
is  promoting  unfavorable  working  relations  among  both  government  agencies 
and  industry.  Certain  accounting  procedures  and  short-sighted  incentives 
which  are  directed  toward  artificial  economic  goals  are  aggravating  the 
effects  of  the  aerospace  depression  and  interfering  with  systems  develop¬ 
ment. 


The  review  of  the  acquisition  process  for  AFSC  test  facilities  resulted 
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in  the  following  observations: 

1.  There  is  a  lack  of  attention  to  the  requirement  for  testing  capa¬ 
bility  to  support  aircraft  development  early  in  the  planning  cycle  for  new 
weapon  systems. 

2.  Acknowledgement  of  test  facility  deficiency  prior  to  the  Program 
Decision  would  likely  reduce  the  probability  of  program  approval. 

g 

3.  The  Military  Construction  Program  (MCP)  has  not  been  and  is  not 
expected  to  be  an  effective  mechanism  for  acquisition  of  major  aerospace 
test  facilities.  The  requirement  that  need  for  a  facility  be  demonstrated 
for  an  "approved"  aircraft  development  program  acts  to  prevent  any  large 
test  facility  construction. 

The  review  of  35  aircraft  development  programs  by  aerospace  experts 
resulted  in  the  following  conclusions  and  appraisals: 

1.  There  was  too  often  a  lack  of  time  and/or  money  early  in  the 
development  program  to  permit  proper  use  of  the  wind  tunnel. 

2.  The  wind  tunnel  test  program  had  a  strong  influence  on  every 
aircraft  development  program  evaluated. 

3.  A  major  shortcoming  in  many  aircraft  developments  was  insuf¬ 
ficient  facility  testing  early  in  the  program  (particularly  prior  to  design 
freeze). 

4.  On  three-fourths  of  the  aircraft  evaluated,  the  experts  estimated 
that  a  more  timely  facility  test  program  and  better  use  of  the  more  sophis¬ 
ticated  test  facilities  would  have  resulted  in: 

a.  Less  overall  development  cost, 

b.  Superior  system  performance,  and 

c.  Less  flight  testing  required. 

5.  Eighty-four  per  cent  of  the  aircraft  experienced  deficiencies  in 
the  flight  evaluation  which  the  experts  believe  might  have  been  minimized 


The  Military  Construction  Program  is  discussed  in  Chapter  V. 
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or  prevented  with  a  more  optimal  facility  test  program. 

Based  upon  the  combined  evaluations  and  opinions  of  the  many  con¬ 
tributing  experts,  a  theory  is  developed  (Chapter  VI)  to  permit  construction 
of  an  optimal  wind  tunnel  test  program  for  future  aircraft  development.  A 
method  for  calculating  the  cost  of  the  facility  test  program  is  included. 

In  order  to  correct  some  of  the  conditions  already  noted,  the  author 
makes  the  following  recommendations: 

1.  The  facility  test  plan  for  system  development  should  be  excluded 
from  the  competition  for  contracts  and  added  to  the  contract  on  a  cost- 
plus-fixed-fee  basis.  Further,  the  AFSC  test  facility  personnel  should 
be  required  to  work  more  closely  with  the  contractor  and  System  Project 
Office  (SPO)*  in  establishing  the  original  test  plan. 

2.  Milestones  which  designate  specifications  and  minimum  accept¬ 
able  performance  criteria  should  be  established  for  the  aerodynamic  portion 
of  each  aircraft  development  program  for  each  major  decision  point.  All 
anticipated  problem  areas  involving  acceptable  data  limits,  data  correction 
techniques,  etc. ,  should  be  resolved  among  the  contractor,  SPO  and  test 
facility  personnel  early  in  the  program  before  a  conflict  arises. 

3.  Complete  facility  test  programs  which  support  aircraft  design 
should  not  be  compromised  when  development  procedures  are  modified  by 
new  ideas. 

4.  Competitive  fly-offs  of  aerodynamic  models  in  the  government 
facilities  should  be  given  strong  consideration  when  full-scale  competi¬ 
tive  fly-offs  are  not  feasible. 

5.  The  government  should  place  increased  emphasis  upon  evalua¬ 
tion  of  the  contractors'  product  in  government  facilities;  the  facilities 
should  be  adequate  to  prevent  misinterpretation  of  the  data  and  argument 
over  the  results. 

6.  AFSC  test  facility  personnel  should  possess  (or  develop)  the 
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expertise  to  advise  the  contractor  and  SPO  of  the  validity  of  the  test 
facility  data  and  to  specify  corrections  to  be  applied  to  the  results.  They 
should  devote  more  effort  to  correlation  of  facility  data  with  flight  results, 

7.  APR  80-14  and/or  AFSC  Supplement  1  to  this  Regulation  should  be 
modified  to  provide  some  separation  of  the  evaluation  and  development  func¬ 
tions.  Freedom  for  test  facility  personnel  to  test  more  independently  and 
advise  the  SPO  (and  AFSC  Commander,  if  necessary)  will  likely  add  greater 
objectivity  to  the  evaluation  procedures. 

8.  The  planning  function  within  AFSC  and  USAF  should  be  revised  to 
make  consideration  of  the  supporting  test  facilities  an  integral  part  of  the 
systems  planning  procedure.  Plans  should  be  established  as  guidelines 
for  action,  with  long  lead-time  items  (like  test  facilities)  receiving  priority 
attention. 

9.  Efforts  should  continue  through  special  committees  and  channels 
to  fund  a  few  major  test  facilities  outside  the  MCP. 

10.  More  permanent  facility  expertise  should  be  placed  in  the  planning 
function  and  MCP  approval  cycle  within  AFSC. 

11.  To  prevent  further  undesirable  proliferation  of  test  facilities  among 
contractors,  the  government  should  establish  a  policy  that  more  clearly 
defines  the  use  of  government  facilities  in  the  development  cycle.  Such  a 
policy  must  provide  boundaries  which  limit  the  competition  among  contrac¬ 
tors  for  bigger  and  better  facilities. 

12.  The  government  must  take  the  lead  in  constructing  new  and  advanced 
test  facilities  where  the  need  is  well  recognized.  Such  a  positive  action 
will  remove  the  likelihood  that  several  similar  facilities  will  be  constructed 
by  competing  industries. 

13.  More  attention  should  be  given  to  improvement  in  testing  techniques 
and  data  interpretation  in  the  existing  facilities. 

14.  The  costs  of  facility  testing  should  be  studied  from  an  Air  Force 
and  national  viewpoint.  Charges  for  use  of  government  facilities  must 
either  be  standardized  or  eliminated.  It  is  recommended  that  service  funding 
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at  AEDC  be  terminated  to  halt  the  very  undesirable  side  effects  until  the 
issue  is  evaluated  in  a  larger  context. 

15.  The  procedures  given  in  this  report  for  construction  of  an  optimal 
facility  test  program  are  recommended  as  a  base  line  for  planning  future 
wind  tunnel  programs. 


CHAPTER  I 


BACKGROUND  AND  INTRODUCTION 

There  is  growing  concern  both  within  the  government  and  private 
industry  over  the  unexpected  increase  in  development  costs  and  reduced 
performance  of  some  recently  developed  weapon  systems.  The  news  media 
has  diligently  provided  sufficient  examples  to  excuse  this  author  from  the 
task  of  listing  them.  However,  the  problem  is  real  and  insiders  in  the 
business  can  probably  supplement  the  published  stories  with  examples 
of  their  own.  None  of  the  military  services  is  exempt  from  criticism,  but 
the  Air  Force  has  received  more  than  its  share  of  attention.  Senator 
McClellan's  investigation  of  the  General  Dynamics  F-111  and  Senator 
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Proxmire's  censure  of  the  Lockheed  C-5A  program  (1)  have  brought  to  a 
focus  various  attempts  to  remedy  the  situation.  One  sometimes  hears  the 
problem  stated  in  the  press  in  terms  of  cost  overrun  or  system  performance 
deficiencies.  Granted,  these  are  the  circumstances  which  cause  all  the 
attention;  but  elimination  of  these  issues  only  cures  the  symptoms  and 
does  not  necessarily  solve  the  problem.  As  most  readers  realize,  the  real 
problem  concerns  the  overall  procedure  for  development  of  technically  pro¬ 
ficient  aerospace  systems  in  the  most  timely  and  economical  manner. 

It  is  not  the  author's  intention  to  ignore  the  substantial  problems 
associated  with  allocation  of  resources  among  the  many  proposed  aerospace 
programs  or,  in  a  broader  context,  allocation  of  resources  among  the  many 
social,  civil  and  military  needs.  However,  the  subject  to  be  addressed 
herein  will  necessarily  be  restricted  to  the  aircraft  system's  development 
process  and  assumes  some  prior  favorable  decisions  in  the  larger  arena. 


Numbers  in  parentheses  refer  to  similarly  numbered  references  in  the 
List  of  References. 
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It  is  not  surprising  that  accusing  fingers  have  been  pointed  in  many 
directions  as  some  officials  search  for  reasons  for  failure  and  others  look 
for  scapegoats.  One  may  hear  a  variety  of  comments  which  place  blame 
on  poor  engineering,  confused  contractor  or  government  management,  in¬ 
adequate  contract,  etc.  The  reader  can  probably  recall  or  perhaps  specify 
other  causes  which  prevented  a  particular  system  from  fulfilling  expecta¬ 
tions,  dependent  upon  his  special  interest  in  the  system's  development. 

Secretary  of  Defense  Laird  addressed  the  problem  directly  on  Feb¬ 
ruary  20,  1970,  when  he  delivered  the  FY  1971  Defense  Program  and 
Budget  to  a  joint  session  of  the  Senate  Armed  Services  Committee  and  the 
Senate  Subcommittee  on  Department  of  Defense  Appropriations  (2).  Secre¬ 
tary  Laird  noted  that  as  of  June  30,  1969,  the  cost  of  34  major  weapon 
systems  had  grown  about  $16.  2  billion  in  excess  of  original  or  baseline 
estimates  previously  reported.  He  stated  that  the  largest  single  cause 
of  cost  growth  is  over-optimism  in  original  cost  estimates.  An  example 
at  that  time  was  the  Air  Force  F-15,  which  had  grown  from  an  original 
estimate  of  about  $6  billion  to  $7.  3  billion  within  about  a  year.  Secretary 
Laird  admitted  that  both  the  contractors  and  military  services  have  the 
same  predispositions  toward  over-optimism  in  estimating  costs.  There  is 
competition  between  programs  for  limited  funds  within  the  services,  and 
the  competition  for  weapon  systems  contracts  stimulates  wishful  thinking 
among  the  contractors.  The  F-15  situation  was  attributed  to  faulty  esti¬ 
mates  in  the  planning  stage.  However,  in  the  F-111  program  the  develop¬ 
ment  problems  were  underestimated  at  the  beginning  of  hardware  develop¬ 
ment  and  the  cost  growth  was  further  increased  because  production  was 
started  before  development  problems  were  solved.  In  general,  there  has 
been  a  tendency  to  cut  the  time  and  effort  which  should  be  spent  in 
development  stages  because  of  failure  to  appraise  adequately  the  risks 
of  major  failures  during  full-scale  development.  Secretary  Laird  identi¬ 
fied  inflation,  engineering  changes,  system  performance  changes,  and 
schedule  changes  as  lesser  reasons  for  cost  growth. 
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The  Deputy  Secretary  of  Defense,  David  Packard,  reported  in  an  inter¬ 
view  (3)  that  almost  every  program  that  is  in  trouble  is  in  such  a  condition 
because  development  has  not  been  done  as  well  as  it  should  have  been,  or 
because  production  was  started  before  development  was  finished.  Dr. 

John  Foster,  Jr. ,  Director  of  Defense  Research  and  Engineering,  presented 
the  FY  1970  Defense  Research,  Development,  Test  and  Evaluation  Program 
(RDT&E)  before  the  House  of  Representatives  Committee  on  the  Armed 
Services  on  April  30,  1969  (4).  In  his  discussion  of  research  and  develop¬ 
ment  management  policies,  he  acknowledged  the  same  criticisms  noted 
by  Messrs.  Laird  and  Packard.  However,  he  reminded  the  committee  of 
the  circumstances  which  evolved  to  bring  about  this  situation.  Over  the 
past  20  years  several  different  development  strategies  have  been  tried. 

It  was  once  popular  to  let  private  contractors  take  all  the  technical  risks, 
support  the  development  costs,  and  offer  the  government  "off-the-shelf" 
hardware.  Unfortunately,  this  policy  did  not  induce  the  contractor  to 
take  risks  to  extend  the  state-of-art  sufficiently  to  produce  the  more 
technically  advanced  systems.  Another  popular  development  scheme  was 
"fly-before-you-buy.  "  Several  contractors  carried  the  system  development 
through  flight  demonstration  at  government  expense  and  the  government 
simply  bought  the  best  one.  The  prime  objection  to  this  technique  was 
the  excessively  large  expense  for  development  of  the  modern  aircraft. 

More  recently  the  systems  development  strategy  has  permitted  selection 
of  a  single  contractor  after  "paper  studies.  "  The  contractor  is  then  held 
accountable  to  produce  within  the  specified  time  and  cost.  Because  costs 
and  time  schedules  are  based  more  on  analyses  than  demonstrated  per¬ 
formance,  there  has  been  a  tendency  for  overruns  and  increased  risks  in 
attempts  to  meet  schedules.  However,  the  management  policies  and 
technical  possibilities  have  not  singularly  shaped  the  development  strategy, 
for  it  has  been  significantly  influenced  by  national  objectives  and  priorities. 
Dr.  Foster  pointed  out  that  the  U.  S.  reaction  to  Soviet  threats,  involve- 
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ment  in  armed  conflicts,  and  variation  in  national  goals  have  had  signifi¬ 
cant  impact  on  the  management  processes.  Dr.  Foster's  comments  serve 
as  a  reminder  that  the  total  environment  has  an  effect  on  the  management 
actions  at  any  given  time.  Further,  when  the  environment  changes  at  a 
rapid  pace,  the  management  practices  may  be  subject  to  criticism  for 
their  slower  response. 

In  testimony  before  congressional  committees  Messrs.  Laird  and 
Packard  and  Dr.  Foster  have  expressed  a  number  of  ideas  for  solution  to 
the  present  dilemma.  It  will  be  noted  that  a  total  change  in  the  develop¬ 
ment  cycle  is  not  proposed;  however,  it  is  recognized  that  certain  pro¬ 
cedures  are  not  being  followed  and  certain  shifts  in  emphasis  are  recom¬ 
mended.  Secretary  Laird  has  noted  the  following  steps  taken  to  reduce 
cost  growth: 

More  realistic  and  accurate  estimates  of  cost  early  in  the  program. 

Better  risk  evaluation  of  the  uncertainties  likely  to  be  encountered 
in  development. 

Emphasis  on  accomplishing  milestones  of  achievement  in  the  develop¬ 
ment  phase  rather  than  meeting  a  predetermined  time  schedule. 

Changes  to  assure  a  minimum  committal  to  production  before 
development  is  complete. 

Steps  to  encourage  better  management  by  both  the  military  service 
and  contractor,  with  more  emphasis  on  meeting  cost  objectives 
rather  than  on  meeting  only  scheduled  and  performance  objectives 

Allowance  for  inflation  in  estimates. 

Reduction  of  dependence  on  paper  analysis  to  validate  designs, 
with  reliance  on  hardware  demonstration  and  competitive  proto¬ 
types  where  feasible.  (2) 

Secretary  Laird  notes,  however,  that  the  high  costs  associated  with 
a  philosophy  of  competitive  "fly-offs"  of  fully-developed  airframes  may 
be  prohibitive  in  most  cases.  He  states  also  that  there  are  general  de¬ 
ficiencies  in  the  amount  and  quality  of  test  and  evaluation  before  systems 
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are  committed  to  full-scale  development  and  production.  The  military 
departments  have  been  instructed  to  identify  and  analyze  the  areas  of 
high  technical  risk,  and  to  defer  system  development  until  risk  analysis 
shows  that  risks  are  acceptable  for  full-scale  development.  Deputy 
Secretary  Packard  (3)  re-emphasized  many  of  the  points  of  Secretary  Laird, 
particularly  in  relation  to  greater  reliance  on  pre-production  hardware 
development  and  evaluation  rather  than  on  paper  studies.  He  contributed 
an  additional  thought  when  he  proposed  keqDing  the  military  project  mana¬ 
ger  and  associated  military  personnel  on  the  program  long  enough  to  make 
a  more  positive  contribution. 

These  prestigious  remarks  on  the  causes  and  remedies  for  cost  over¬ 
runs  and  system's  performance  deficiencies  demonstrate  that  there  is 
pressure  to  correct  the  situation.  However,  it  is  obvious  to  many  that 
most  of  the  steps  proposed  for  correction  do  not  differ  substantially  from 
the  established  development  directives  and  that  there  would  have  been  no 
problem  if  they  had  been  followed  in  the  first  place.  There  are  apparently 
strong  counter  influences  and  incentives  that  are  preventing  implementation 
of  these  procedures. 

Instructions  to  "produce  better  cost  estimates"  or  "provide  better 
management"  do  not  provide  tangible  directives  for  those  trying  to  make  a 
change.  Nor  is  it  clear  how  to  implement  guidelines  which  specify  addi¬ 
tional  emphasis  on  problem  solving  early  in  the  program  in  order  to  reduce 
risks.  How  is  this  to  be  effected  within  financial  and  time  constraints? 

The  technology  must  be  advanced  to  the  system  stage  fairly  rapidly  or  it 
will  be  obsolete  before  it  is  developed.  Where  is  the  development  system 
failing?  Where  specifically  should  emphasis  be  put?  How  are  the  rather 
general  statements  of  problem  solution  to  be  implemented?  In  order  to 
answer  these  questions,  one  must  take  a  critical  look  at  the  test  and 
evaluation  program.  The  author  asserts  that  it  is  deficiencies  in  this  pro¬ 
gram,  coupled  with  varying  interpretations  and  understandings  of  its  imple¬ 
mentation,  that  present  one  of  the  more  basic  problems  in  aircraft  develop- 
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ment. 

Military  research  and  development  is  evolved  in  a  complex  and  con¬ 
fusing  world;  forces  and  counter-forces  struggle  for  domination.  The 
reader  will  recognize  competition  among  the  various  technologies  and 
approaches  for  any  particular  aircraft  development;  there  are  also  compet¬ 
ing  development  strategies.  A  conflict  usually  exists  between  the  desire 
for  technically  advanced  weapons  systems  and  economic  limitations.  This 
environment  is  not  likely  to  change,  and  to  a  certain  extent  may  be  con¬ 
sidered  healthy.  However,  there  are  other  forces  which  add  to  this  con¬ 
fusion.  The  professional  men  within  the  government  who  are  charged  with 
responsibility  for  various  facets  of  a  system's  development  are  usually 
highly  motivated  toward  job  accomplishment.  An  examination  of  their 
working  environment  will  show  that  they  are  sometimes  put  in  job  situations 
where  their  personal  goals  may  not  be  best  served  by  devotion  to  job- 
oriented  goals  and,  in  a  larger  context,  where  the  defined  job-oriented 
goal  is  in  conflict  with  a  more  important  total  objective.  Such  a  situation 
is  neither  healthy  nor  necessary. 

This  study  attempts  recognition  of  the  many  influences  on  the  aircraft 
development  program  and  on  the  individuals  involved  as  it  seeks  to  place 
environmental  test  facilities  in  perspective  in  the  development  strategy 
for  aeronautical  systems.  The  author  is  aware  that  he  has  isolated  a 
small  part  of  a  complicated  process  for  investigation;  lack  of  recognition 
of  other  steps  and  emphasis  in  the  development  cycle  should  not  be  inter¬ 
preted  as  downgrading  their  importance.  However,  the  author  does  seek 
to  convince  the  reader  that  inattention  to  the  proper  use  of  test  facilities 
in  the  development  cycle  is  causing  problems  with  substantial  detrimental 
effects.  The  reader  may  observe  that  these  detrimental  effects  are  often 
attributed  to  other  causes,  and  the  problems  remain  unsolved. 

The  author  has  turned  to  the  practicing  experts  to  test  his  hypotheses 
and  gain  new  information.  The  data  presented  herein  have  been  collected 
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primarily  from  questionnaires  which  have  been  completed  by  many  of  the 
leading  people  in  the  aerospace  business.  About  one-half  of  these  ex¬ 
perts  have  also  been  interviewed  by  the  author  to  assure  accurate  inter¬ 
pretation  of  answers  and  to  expand  key  arguments.  The  respondents  listed 
in  Appendix  A  represent  a  substantial  portion  of  this  country's  knowledge 
and  experience  in  aircraft  development.  These  individuals,  both  from  the 
industry  and  government,  have  been  instrumental  in  previous  aircraft 
development  programs  and  are  actively  making  major  decisions  today. 

What  they  think  and  why  they  believe  as  they  do  is  real  and  important  data, 
whether  or  not  one  agrees  with  individual  opinions.  The  reader  is  cautioned 
to  look  for  those  situations  in  the  following  discussions  where  honest  dif¬ 
ferences  of  opinion  are  causing  problems.  The  reader  is  also  asked  to 
observe  areas  of  overwhelming  agreement  among  the  experts  and  (as  the 
author  has)  carefully  evaluate  his  own  thoughts  when  they  do  not  agree. 

It  is  not  the  intent  of  this  report  to  tell  the  "true  story"  and  straighten-out 
the  biased  reader.  Instead,  the  author  wishes  the  reader  to  gain  new  in¬ 
formation  which  will  permit  him  to  make  adjustments  in  his  own  thinking 
and  in  established  procedures,  and  in  turn  to  improve  the  aircraft  develop¬ 
ment  process.  The  knowledgeable  reader  will  probably  be  able  to  deduct 
subtleties  not  included  in  the  short  discussions.  It  is  hoped  that  these 
data  will  have  usefulness  far  outside  the  bounds  of  this  study. 

Chapter  II  in  this  report  is  designed  to  give  general  information  about 
the  data  and  its  adequacy.  The  two  forms  of  the  questionnaire  are  discus¬ 
sed,  and  the  reader  is  guided  through  the  questionnaires  with  comments 
on  special  objectives  and  terminology.  The  aerospace  experts  who  com¬ 
pleted  the  questionnaires  and  granted  interviews  to  the  author  are  further 
classified  and  their  qualifications  are  noted. 

Chapter  III  addresses  the  origin  of  the  facility  test  plan  and  questions 
the  applicability  of  certain  incentives  and  Air  Force  Regulations.  It  is 
hypothesized  that  competitive  procedures  designed  to  select  contractors 
and  reduce  aircraft  development  costs  and  time  are  causing  adverse  effects 
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on  the  facility  test  plan.  An  attempt  is  made  to  define  the  contribution  of 
various  participating  groups  to  the  test  plan.  Changes  in  the  present 
system  of  originating  a  facility  test  plan  are  evaluated  and  discussed. 

The  role  of  the  test  facility  in  the  systems  development  and  acquisi¬ 
tion  cycle  is  explored  in  Chapter  IV,  and  some  shift  in  emphasis  is  sug¬ 
gested.  It  is  hypothesized  that  the  full  intent  of  the  DoD  development 
directives  is  not  being  covered  by  present  practices.  It  is  further  hypo¬ 
thesized  that  the  test  facility  can  and  should  play  an  increased  role  in 
the  implementation  of  the  new  development  policy.  Very  specific  examples 
are  used  to  show  how  the  facility  can  provide  milestone  checks  and  pro¬ 
duce  information  to  support  important  program  decisions. 

Chapter  V  contains  a  study  of  the  adequacy  of  test  facilities  to  accom¬ 
plish  the  desired  testing  and  an  Investigation  of  procedures  to  permit  im¬ 
provement  in  facility  capability.  It  is  hypothesized  that  inadequacies  in 
the  test  facilities  are  causing  detrimental  effects  in  systems  development, 
and  that  this  trend  toward  facility  obsolescence  is  getting  worse.  The 
roles  of  the  government  and  contractor  test  facilities  are  explored  and 
evaluated  in  an  environment  which  includes  a  declining  test  workload. 

Chapter  VI  includes  a  historical  review  of  35  aircraft  development 
programs  by  experts.  It  is  hypothesized  that  a  critical  review  of  the  use 
and  misuse  of  the  wind  tunnel  in  many  completed  development  programs 
will  permit  formulation  of  a  theory  for  more  optimal  use  of  the  test  facility. 
A  multiple  regression  analysis  is  used  to  define  "predictors"  which  will 
allow  characterization  of  an  optimal  wind  tunnel  program  from  basic  know¬ 
ledge  about  the  proposed  aircraft.  Such  information  is  considered  useful 
as  a  first  step  in  planning  new  test  programs  and  should  be  helpful  to  those 
who  attempt  to  predict  future  workloads  in  test  facilities.  The  reader  may 
be  interested  in  the  testimony  of  the  expert  evaluators  which  is  used  to 
evaluate  the  effectiveness  of  past  wind  tunnel  programs. 

The  author  has  already  become  aware  that  an  attempt  at  a  scholarly 


19 


approach  to  such  a  complicated  process  is  likely  to  draw  criticism  from 
those  who  consider  themselves  the  real  practitioners.  Everyone  who  has 
been  associated  with  any  phase  of  the  aircraft  development  cycle  has  had 
the  opportunity  to  mold  his  own  opinion  based  upon  his  unique  experiences; 
the  author  is  no  exception.  To  some  readers,  the  whole  study  may  appear 
"slanted"  because  it  does  not  address  what  they  consider  to  be  "really 
important"  from  their  own  experiences.  Some  readers  may  feel  that  "their 
side  of  an  argument"  is  not  presented  in  sufficient  detail.  The  author 
admits  brevity  on  both  sides  of  most  arguments;  however,  an  attempt  is 
made  to  explain  minority  opinion. 

These  analyses  may  fall  short  of  confirming  given  hypotheses  for  some 
readers.  Comparisons  of  historical  events  in  aircraft  development  programs 
are  not  only  difficult,  but  risky  because  of  uncontrollable  extraneous  var¬ 
iables.  Unfortunately,  this  task  does  not  lend  itself  to  a  laboratory  envi¬ 
ronment  where  parameters  can  be  controlled  at  will.  Further,  value  judg¬ 
ments  are  an  essential  part  of  the  data,  and  value  has  a  meaning  which  is 
personal.  Conclusions  based  upon  such  qualitative  diagnoses  cannot  be 
proven  to  an  impartial  observer.  Under  such  circumstances,  the  closest 
one  can  hope  to  approach  "hard  data"  is  the  thoughtful  opinions  of  exper¬ 
ienced  and  expert  personnel.  The  author  hopes  that  some  of  the  comments 
and  recommendations  in  this  report  will  benefit  the  government  decision¬ 
maker  and  policy-maker. 


CHAPTER  II 


DISCUSSION  OF  THE  RESPONDENTS  AND  QUESTIONNAIRES 

A  large  portion  of  the  information  of  interest  in  this  study  is  composed 
of  the  observations  and  evaluations  of  knowledgeable  and  experienced 
personnel  in  the  aerospace  business.  The  questionnaire  was  selected  as 
the  most  appropriate  route  to  reach  a  large  and  diversified  audience  in  a 
limited  period  of  time;  however,  the  author  did  perform  some  follow-up 
interviews  with  the  respondents  to  obtain  more  complete  information.  Ail 
of  the  industry  experts  who  evaluated  the  historical  aircraft  development 
programs  were  personally  interviewed.  The  questionnaires  were  completed 
and  interviews  conducted  during  the  period  between  early  February,  1971, 
and  mid-May,  1971.  Over  50%  of  the  questionnaires  were  answered  and 
returned  to  the  author  for  inclusion  in  this  report.  The  responses  which 
were  received  after  May  15,  1971,  are  not  included  in  the  tabulated  data, 
although  the  contributor  is  Identified  along  with  the  others  in  Appendix  A. 

An  attempt  was  made  to  survey  a  wide  spectrum  of  experts  from  private 
Industry  and  government.  Of  the  117  questionnaires  returned,  58  are  from 
industry  and  57  from  various  government  organizations.  The  author  inter¬ 
prets  such  a  substantial  response  to  an  admittedly  lengthy  questionnaire 
by  such  a  prestigious  group  of  respondents  as  evidence  of  the  aerospace 
community's  interest  in  some  of  the  subjects  under  review.  The  industry 
respondents  represent  the  following  companies: 

The  Boeing  Company 

Fairchild  Hiller/Republic  Division 

General  Dynamic s/Convair  Division  (San  Diego  and  Fort  Worth) 

Grumman  Aerospace  Corporation 

LTV  Aerospace/Vought  Aeronautics  Division 
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Lockheed  Aircraft  Corporation/ Georgia  Company 
McDonnell  Aircraft  Company 

North  American  Rockwell  Corporation  (Los  Angeles  &  Columbus) 
Northrop  Corporation 

In  many  cases  questionnaires  were  also  returned  from  company  corporate 
officers. 

The  government  respondents  represent  several  organizations  that  con¬ 
tribute  to  various  aspects  of  the  aircraft  development  cycle.  Eighteen  ques¬ 
tionnaires  were  returned  by  personnel  of  the  Aeronautical  Systems  Division, 
Wright-Patterson  Air  Force  Base,  Ohio.  Within  this  group  is  located  the 
System  Project  Office  and  System  Project  Director  (SPD):  *  centered  here 
is  the  major  responsibility  for  aircraft  development  within  the  Air  Force. 

Ten  questionnaires  were  completed  by  members  of  the  Air  Force  Systems 
Command  Laboratories,  also  located  at  Wright-Patterson  Air  Force  Base; 

8  respondents  were  from  the  Flight  Dynamics  Laboratory  and  2  from  the  Aero 
Propulsion  Laboratory.  Four  responses  were  received  from  Headquarters, 

Air  Force  Systems  Command,  and  3  were  received  from  Headquarters,  U.  S. 
Air  Force.  These  7  responses  are  grouped  together  and  will  be  referenced 
as  "Headquarters.  "  The  Department  of  Defense  Test  Center  is  represented 
by  the  Arnold  Engineering  Development  Center.  These  13  responses  are 
tabulated  together,  although  7  are  from  Air  Force  personnel  and  6  are  from 
personnel  of  the  AEDC  operating  contractor,  ARC,  Inc.  Eight  questionnaires 
were  returned  by  members  of  the  National  Aeronautics  and  Space  Administra¬ 
tion;  5  came  from  Langley  Research  Center,  2  from  Ames  Research  Center, 
and  1  from  Lewis  Research  Center.  Five  questionnaires  were  returned  by 
members  of  the  USAF  Scientific  Advisory  Board,  Aerospace  Vehicles  Panel. 
However,  1  of  these  gentlemen  is  with  NASA  and  2  are  with  private  industry, 
so  their  responses  are  included  in  their  respective  groups.  The  other  2 
board  members  are  located  in  universities.  These  2  responses,  along  with 
1  questionnaire  which  was  returned  without  identification,  will  make  up  the 
group  designated  as  "others.  " 
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Two  different  questionnaire  forms  were  used  in  the  survey,  although 
some  of  the  questions  are  similar.  Appendix  B  contains  the  shorter  ques¬ 
tionnaire  which  was  used  throughout  the  government  and  part  of  industry. 

It  is  an  opinion-attitude  survey  and  asks  the  respondent  to  express  his 
beliefs  and  relate  his  experiences  concerning  certain  procedures  and 
practices  associated  with  the  role  of  the  test  facility  in  the  aircraft 
development  cycle.  Appendix  C  contains  the  first  25  questions  of  the 
longer  questionnaire.  The  remainder  of  this  questionnaire  (Questions  26 
through  47)  is  identical  with  Questions  1  through  22  in  Appendix  B.  The 
questionnaire  of  Appendix  C  was  directed  to  those  respondents  in  industry 
who  were  evaluating  particular  aircraft  development  histories. 

The  questionnaire  of  Appendix  B  was  sent  to  a  selected  group  of  indi¬ 
viduals  within  government  and  the  USAF  Scientific  Advisory  Board.  The 
author  was  seeking  the  more  experienced  and  senior  personnel  who  had 
worked  in  various  areas  of  aircraft  development.  The  author  depended 
upon  certain  respected  individuals  within  each  group  to  recommend  others 
who  possessed  the  desired  qualifications.  Different  viewpoints  were  al¬ 
most  assured  by  selection  of  personnel  from  the  various  government  organi¬ 
zations.  Some  of  these  shorter  questionnaires  were  also  sent  to  contacts 
in  each  of  the  participating  aircraft  companies,  along  with  instructions 
that  they  were  for  the  knowledgeable  and  senior  personnel  who  would  not 
have  the  opportunity  to  complete  the  longer  questionnaire. 

The  first  9  questions  in  Appendix  B  are  directed  toward  an  investiga¬ 
tion  of  the  conception  and  development  of  wind  tunnel  test  programs. 
Questions  10  through  14  deal  with  the  role  of  the  test  facility  as  it  relates 
to  the  new  DoD  development  policy.  Questions  15  through  20  and  22  are 
concerned  with  the  adequacy  of  facilities  to  provide  the  required  test  en¬ 
vironment,  the  effects  of  inadequacies,  and  possible  remedies.  Question 
21  is  directed  toward  improvement  in  operating  procedures  for  DoD  test 
facilities.  As  noted,  these  questions  were  on  both  questionnaires.  The 
shorter  questionnaire  is  completed  with  5  questions  which  were  general- 
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ized  from  queries  on  specific  development  programs  on  the  longer  ques¬ 
tionnaire. 

The  evaluators  who  completed  the  longer  questionnaire  (Appendix  C) 
were  individuals  who  had  been  closely  associated  with  the  particular 
aircraft  development  program.  Because  the  wind  tunnel  test  program  is 
only  one  of  several  variables  which  influence  the  success  or  failure  of  a 
particular  aircraft  development,  only  those  individuals  who  were  on  the 
scene  and  making  the  history  are  in  a  position  to  separate  the  variables 
and  distinguish  among  the  influences.  The  author  asked  for  personnel  who 
had  served  in  responsible  positions  during  the  development  program.  Most 
of  the  respondents  were  either  Project  Aerodynamicists  or  Engineering  Man¬ 
agers.  Sometimes  aircraft  were  omitted  from  the  survey  because  appropriate 
evaluators  were  no  longer  with  the  company.  It  was  originally  intended  to 
have  several  evaulations  on  each  aircraft,  but  this  was  possible  in  only  a 
few  cases.  The  first  12  questions  of  the  longer  questionnaire  deal  with 
historical  data  on  the  use  of  the  test  facility  in  the  development  program. 

Of  particular  interest  is  the  quantity  of  testing,  type  of  test  facility,  timing 
of  the  tests  in  the  development  cycle,  and  cost  of  testing.  Questions  13 
through  18  are  an  evaluation  of  that  particular  test  program.  The  reviewers 
were  then  asked  to  make  changes  in  the  wind  tunnel  test  program  that  would, 
in  their  opinions,  have  made  it  more  effective;  they  were  required  to  weigh 
the  technical  benefits  of  testing  against  the  expenditure  of  time  and  money 
to  achieve  the  testing.  This  revised  test  program  is  referred  to  as  "optimal" 
in  the  text. 

It  is  suggested  that  the  reader  take  note  of  the  preamble  to  Questions  4 
and  19  in  Appendix  C.  The  former  describes  the  type  of  wind  tunnel  tests 
to  be  included  in  the  review.  The  "wind  tunnel  test  program"  includes  all 
aero-dynamic  and  structural  testing  in  wind  tunnels;  propulsion-aircraft 
integration  type  tests  are  included.  Development  of  propulsion  systems 
is  excluded.  The  questionnaire  differentiates  between  testing  accomplished 
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in  support  of  original  development  and  that  required  by  retrofit  programs 
or  follow-on  versions  of  the  same  aircraft.  Engine  development  testing 
was  excluded  from  this  part  of  the  survey  to  limit  the  scope  of  the  study, 
since  it  would  have  involved  a  different  group  of  contractors  and  evalua¬ 
tors.  The  paragraph  preceding  Question  19  is  designed  to  establish 
criteria  for  determination  of  the  optimal  test  program.  The  reader  is  ad¬ 
vised  to  read  this  section,  for  it  is  important  to  the  understanding  of  the 
evaluator's  objective  in  establishing  an  optimal  test  program. 

The  historical  data  on  the  facility  test  programs  were  difficult  for  some 
respondents  to  produce.  Some  companies  keep  this  information  in  more 
retrievable  form  than  others;  some  respondents  had  more  opportunity  or 
were  more  motivated  to  search  for  the  data  than  others.  There  are  gaps 
and  uncertainties  in  the  data  which  generally  Increase  with  the  age  of  the 
aircraft  development  program.  The  cost  data  in  particular  is  sketchy;  in 
several  cases  it  was  just  not  available.  Furthermore,  the  charges  to  a 
particular  category  varied  from  one  company  to  the  next.  Less  than  25% 
of  the  questionnaires  have  complete  cost  data  which  is  compatible  with 
the  instructions  in  the  questionnaire.  However,  the  author  was  pleased 
with  the  overall  integrity  of  the  information,  for  it  permitted  pursuit  of 
most  of  the  intended  analyses. 

The  author  was  particularly  pleased  with  the  narrative  responses  to 
questions  on  both  questionnaires.  Many  respondents  expanded  their  ans¬ 
wers  on  added  pages.  The  author  regrets  that  he  is  unable  to  quote  more 
of  the  excellent  replies.  However,  for  the  sake  of  conciseness,  an  attempt 
has  been  made  to  summarize  the  answers  and  quote  only  the  representative 
majority  and  minority  opinions. 

It  may  be  noted  in  the  cover-letters  to  the  respondents  (Appendices  B 
and  C)  that  the  respondent  is  assured  a  degree  of  anonymity  of  his  own 
choosing.  Most  of  the  respondents  accepted  the  proposition  offered  in  the 
cover  letter;  i.  e. ,  recognition  as  a  contributor  but  no  identification  with 
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particular  remarks  or  opinions.  One  respondent  asked  not  to  be  identified, 
and  several  replied  that  they  might  be  quoted.  The  author  has  honored 
this  agreement  by  avoiding  comments  with  associations  which  "insiders" 
could  attribute  to  individual  respondents. 


CHAPTER  III 


THE  ORIGIN  OF  THE  WIND  TUNNEL  TEST  PROGRAM 

The  wind  tunnel  test  program  which  supports  the  development  of  a 
particular  aircraft  system  is  conceived  and  implemented  in  accordance 
with  Air  Force  Regulation  No.  80-14  and  AFSC  Supplement  1  to  that  regu¬ 
lation  (5  and  6).  It  is  assumed  that  most  readers  are  familiar  with  these 
documents;  for  those  who  are  not,  a  few  words  of  explanation  should  suf¬ 
fice  for  this  discussion.  The  scope  of  the  referenced  documents  is  broader 
than  just  the  facility  test  program,  and  encompasses  ail  testing  (including 
flight  testing)  for  all  phases  of  the  test  program,  thus  covering  the  life 
cycle  of  the  system  from  its  beginning  through  the  acquisition  phase.  How¬ 
ever,  the  interest  in  this  study  is  in  that  portion  of  the  documents  which 
is  connected  with  development  and  implementation  of  the  facility  test  plan. 

The  responsible  development  organization  (usually  ASD),  acting  through 
the  System  Project  Office,  is  directed  via  the  regulations  to  provide  for 
the  development  of  the  System  Test  Plan.  The  individual  test  organizations 
within  the  AFSC  are  directed  to  provide  certain  advice  and  support  to  the 
developing  organization  to  the  extent  requested  by  the  developing  organi¬ 
zation.  It  is  standard  practice  for  a  potential  contractor  (bidder)  to  submit 
a  rather  detailed  test  plan  for  wind  tunnel  testing  as  part  of  his  response 
to  the  invitation  to  bid  and  in  accordance  with  guidelines  in  the  invitation. 
The  bidder's  test  plan  is  thus  evaluated  along  with  the  remainder  of  his 
proposal  and  exerts  some  influence  upon  the  acceptance  of  that  proposal. 

In  this  chapter,  the  adequacy  of  the  established  procedure  for  origi¬ 
nation  of  the  wind  tunnel  test  plan  is  questioned.  It  is  hypothesized  that 
conflicting  incentives  tend  to  reduce  the  scope  and  effectiveness  of  the 
test  plan.  An  investigation  is  made  to  determine  the  influence  on  the  test 
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plan  by  certain  parties  within  the  government  who  are  in  positions  to  make 
a  contribution.  Finally,  some  alternatives  for  changes  inthe  existing  pro¬ 
cedures  are  suggested  and  evaluated. 

There  are  no  "right"  or  "wrong"  replies  to  these  questions,  for  they 
only  indicate  the  respondent's  belief  in  the  truth  of  the  proposition.  As 
several  respondents  have  observed,  a  definite  answer  would  require  ref¬ 
erence  to  a  particular  example;  otherwise,  there  will  always  be  exceptions. 
The  author  had  recognized  this  situation  and  the  respondent  was  instructed 
to  base  his  answers  on  his  overall  experience  and  observations.  The 
strength  of  his  agreement  or  disagreement  should  then  be  some  measure  of 
the  validity  of  the  hypothesis. 

Factors  that  Influence  the  Facility  Test  Plan 

It  is  speculated  that  the  emphasis  on  low  development  cost  and  a 
short  time  period  to  produce  a  flight  aircraft  has  the  effect  of  encouraging 
the  prospective  contractor  to  reduce  the  scope  of  his  proposed  test  pro¬ 
gram  and  thus  increase  his  risks.  Few  will  deny  that  such  emphasis  is 
indeed  a  factor  in  most  development  programs.  The  military  developing 
agency  has  usually  specified  a  date  for  aircraft  operation  which  dictates 
a  rather  hurried  development  program,  and  the  prospective  contractor  is 
keenly  aware  that  development  funds  are  limited. 

In  order  to  agree  with  this  hypothesis,  the  respondent  must  acknow¬ 
ledge  that  both  the  pressures  of  time  and  cost  act  to  force  the  contractor 
to  reduce  his  proposed  facility  test  plan.  Further,  the  respondent  must 
agree  that  reduction  in  facility  test  programs  is  associated  with  increased 
technical  risk.  This  premise  is  addressed  in  Question  1,  Appendix  B,  and 
the  results  of  the  survey  are  noted  in  Table  1.  It  is  clear  that  most  of  the 
experts  agree  with  the  hypothesis  with  87%  agreeing  and  10%  disagreeing. 
Thirty-one  percent  were  motivated  to  agree  strongly.  The  conclusion  to 
be  drawn  from  this  response  is  that  about  9  out  of  10  of  the  people  surveyed 
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acknowledge  that  pressures  associated  with  development  cost  and  time 
do  exist  and  force  one  in  the  direction  of  less  testing;  further,  a  reduced 
test  facility  program  is  associated  with  increased  technical  risk. 


TABLE  1 

REPLIES  TO  HYPOTHESIS  THAT  EMPHASIS  ON  LOW 
DEVELOPMENT  COST  AND  SHORT  DEVELOPMENT 
TIME  ENCOURAGES  THE  BIDDER  TO  REDUCE 
THE  SCOPE  OF  THE  FACILITY  TEST  PROGRAM 
AND  INCREASE  HIS  RISKS 


Agency 

Disagree 

Strongly 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

1 

8 

1 

34 

14 

ASD 

0 

3 

1 

10 

4 

Headquarters 

0 

0 

0 

4 

3 

AFSC  Labs 

0 

0 

1 

7 

2 

AEDC 

0 

0 

0 

3 

10 

NASA 

0 

0 

0 

5 

3 

Other 

0 

0 

0 

3 

0 

Totals 

1 

11 

3 

66 

36 

It  is  suggested  that  the  developing  agency  and/or  bidder  have  a 
tendency  to  minimize  attention  to  the  test  plan  in  the  bid  proposal  because 
of  the  many  other  items  they  may  consider  more  important.  For  example, 
emphasis  on  aircraft  performance,  cost  proposals,  development  schedules, 
etc.  may  not  permit  the  bidder  or  evaluator  to  direct  much  attention  to  the 
adequacy  of  the  test  plan  at  this  point  in  time.  This  supposition  is  ad¬ 
dressed  in  Question  2,  Appendix  B,  and  the  response  is  summarized  in 
Table  2.  The  hypothesis  is  affirmed  by  a  small  majority,  with  about  55% 
in  agreement  and  39%  opposed.  However,  some  stratification  of  opinion 
among  the  different  groups  might  be  noted  here.  The  industry  personnel 
reject  the  hypothesis  by  about  the  same  ratio  that  the  total  group  accepts 
it.  Many  of  the  industry  respondents  are  responsible  for  development  of 


the  test  plan  within  their  respective  companies.  The  AEDC  and  NASA 
respondents,  who  are  test  oriented,  are  very  much  in  agreement  with  the 
hypothesis  (86%  agreement). 
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TABLE  2 

REPLIES  TO  HYPOTHESIS  THAT  A  TENDENCY 
EXISTS  TO  MINIMIZE  ATTENTION  TO  THE 
TEST  PLAN  BECAUSE  OF  THE  MANY  OTHER 
ISSUES  CONSIDERED  MORE  IMPORTANT  IN 
THE  BID  PROPOSAL 


Agency 

Disagree 

Strongly 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

3 

29 

2 

23 

1 

ASD 

1 

6 

3 

7 

1 

Headquarters 

0 

1 

0 

3 

3 

AFSC  Labs 

0 

3 

1 

6 

0 

AEDC 

0 

1 

0 

6 

6 

NASA 

0 

1 

1 

6 

0 

Other 

0 

1 

0 

2 

0 

Totals 

4 

42 

7 

53 

11 

It  is  speculated  that  many  of  the  personnel  involved  in  the  prepara¬ 
tion  and  evaluation  of  the  bid  may  feel  that  the  test  plan  in  the  potential 
contractor's  proposal  has  little  influence  on  the  evaluation  of  the  bid  and 
thus  has  little  effect  upon  selection  of  the  contractor.  Question  3,  Appen¬ 
dix  B,  is  directed  to  an  evaluation  of  this  premise  and  the  results  are  listed 
in  Table  3.  Overall,  the  hypothesis  is  accepted;  46%  affirm  and  33% 
reject.  Evidently,  there  is  substantial  belief  on  both  the  part  of  the 
industry  bidder  and  government  evaluator  that  the  facility  test  plan  in  the 
proposal  carries  little  weight — at  least  more  often  than  not. 

The  replies  to  these  first  three  questions  lead  one  to  believe  that 
the  competitive  nature  of  the  bidding  procedure  hinders  rather  than  helps 
the  goal  of  achieving  a  good  facility  test  plan.  One  might  conclude  that 
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the  system  acts  to  force  test  plans  toward  inadequacy  because: 

—  They  are  not  really  important  in  deciding  who  gets  a  contract. 

—  Time  and  effort  spent  in  preparing  and  evaluating  the  contract 
is  better  directed  to  more  important  issues. 

—  The  usual  emphasis  oh  low  development  cost  and  short  develop¬ 
ment  time  encourages  a  potential  contractor  to  accept  increased 
risks  through  minimization  of  his  test  plan. 


TABLE  3 

REPLIES  TO  HYPOTHESIS  THAT  THE  MAGNITUDE 
AND  QUALITY  OF  THE  TEST  PLAN  IN  THE  BID 
PROPOSAL  HAS  LITTLE  INFLUENCE  ON  THE 
EVALUATION  OF  THE  BID  AND  SELECTION 
OF  THE  CONTRACTOR 


Agency 


Disagree  No  Agree 

Strongly  Disagree  Opinion  Agree  Strongly 


Industry  0 

ASD  1 

Headquarters  0 

AFSC  Labs  2 

AEDC  1 

NASA  1 

Other  0 

Totals  5 


19 

7 

2 

0 

1 

4 

_J, 

34 


16  20  3 

2  6  2 

1  3  1 

2  5  1 

0  8  3 

3  0  0 

1  1  _g 

25  43  10 


The  interpretation  of  these  questions  and  answers  was  substantiated 
in  the  personal  interviews.  Another  interesting  point  was  acknowledged  by 
several  individuals  in  the  interviews.  Because  of  the  stress  on  low  risk 
development  programs,  the  contractor  tries  to  propose  the  best  system  per¬ 
formance  with  fewest  unproven  technical  advances.  He  may  feel  that  pro¬ 
posing  a  substantial  facility  test  program  for  any  one  technical  area  will 
indicate  his  uncertainty  in  that  technical  area  and  will  work  against  accept¬ 
ance  of  his  proposal. 
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Thus,  it  appears  that  the  incentives  introduced  in  the  process  for 
selection  of  a  contractor  are  resulting  in  acceptance  of  increased  risk 
before  the  development  program  even  begins.  It  is  to  be  expected  that  the 
individuals  involved  in  the  preparation  and  evaluation  of  the  proposal  will 
direct  their  attention  to  those  items  that  they  feel  are  most  influential  in 
determining  the  winner  of  the  contract.  It  has  been  demonstrated  that  this 
emphasis  has  a  natural  side  effect;  i.  e. ,  less  attention  to  the  origination 
of  the  facility  test  plan.  The  far-reaching  consequences  of  an  inadequate 
test  program  are  discussed  in  later  chapters.  However,  it  is  important  to 
note  here  that  the  incentive  system  as  applied  to  test  plan  origination  is 
achieving  exactly  opposite  results  from  those  desired;  i.  e. ,  increased 
risks  and  the  possibility  of  overall  increased  development  costs. 

Contributors  to  the  Test  Plan 

There  are  several  parties  who  are  associated  with  the  origination  of 
the  test  plan.  It  is  the  intent  of  the  following  analysis  to  determine  the 
degree  of  influence  various  groups  have  on  the  magnitude  and  quality  of 
the  facility  test  plan.  The  military  developing  organization  (ASD)  is  re¬ 
sponsible  for  the  development  of  the  test  plan  by  regulation;  however,  it 
may  seek  help  and  advice  from  other  government  agencies.  It  is  of  interest 
here  to  measure  just  how,  much  influence  each  party  exerts  on  the  facility 
test  plan  by  his  own  estimate  and  that  of  others. 

It  is  speculated  that  the  contractor  is  the  most  influential  figure  in 
the  development  of  the  facility  test  plan.  It  is  he  who  prepares  the  test 
plan  as  part  of  his  bid  proposal.  This  hypothesis  is  addressed  in  Question 
4,  Appendix  B,  and  the  replies  are  noted  in  Table  4. 


TABLE  4 


REPLIES  TO  HYPOTHESIS  THAT  THE  CONTRACTOR 
DOES  MOST  OF  THE  WORK  IN  PREPARING 
THE  FACILITY  TEST  PLAN  FOR  SYSTEMS' 
DEVELOPMENT 


Agency 

Disagree 

Strongly 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

0 

4 

1 

44 

9 

ASD 

0 

3 

1 

13 

1 

Headquarters 

0 

1 

2 

3 

1 

AFSC  Labs 

0 

0 

0 

9 

1 

AEDC 

0 

0 

1 

6 

6 

NASA 

0 

0 

2 

6 

0 

Other 

0 

0 

1 

2 

0 

Totals 

0 

8 

8 

83 

18 

The  hypothesis  is  strongly  accepted;  86%  agree  and  7%  disagree.  Evi¬ 
dently,  the  contractor  is  a  very  important  figure  in  the  development  of  the 
test  plan. 

Since  the  contractor  plays  such  an  important  role  in  origination  of 
the  test  plan,  it  is  desirable  to  see  how  much  influence  the  responsible 
organization  (ASD)  exerts  on  the  size  and  quality  of  the  test  plan.  Ques¬ 
tion  5,  Appendix  B,  is  worded  to  suggest  little  influence,  and  the  results 
are  tabulated  in  Table  5.  Overall,  the  hypothesis  is  rejected  with  56% 
disagreeing  and  36%  agreeing.  However,  if  the  ASD  responses  are  removed 
from  consideration,  the  proposition  is  rejected  by  only  a  50%-42%  margin. 

It  would  appear  that  other  respondents  do  not  agree  with  the  developing 
organization  on  the  extent  of  influence  exerted,  since  ASD  rejected  the 
hypothesis  by  a  ratio  of  95%  to  5%. 
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TABLE  5 

REPLIES  TO  HYPOTHESIS  THAT  THE  MILITARY 
DEVELOPING  AGENCY  EXERTS  VERY  LITTLE 
INFLUENCE  ON  THE  MAGNITUDE  OR 
QUALITY  OF  THE  TEST  PLAN 


Agency 

Disagree 

Strongly 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

0 

33 

4 

18 

3 

ASD 

2 

15 

0 

1 

0 

Headquarters 

1 

3 

0 

2 

1 

AFSC  Labs 

1 

3 

1 

5 

0 

AEDC 

0 

3 

1 

6 

3 

NASA 

0 

3 

3  ' 

2 

0 

Other 

0 

2 

0 

1 

0 

Totals 

4 

62 

9 

35 

7 

The  DoD  facility  test  personnel  (primarily  AEDC)  also  participate  in 
the  origination  of  the  facility  test  plan;  however,  their  participation  is 
controlled  to  a  large  extent  by  invitation  from  ASD.  The  extent  of  AEDC's 
influence  on  the  magnitude  and  quality  of  the  facility  test  plan  is  explored 
in  Question  6,  Appendix  B,  and  the  responses  are  shown  in  Table  6.  Sixty- 
one  per  cent  accept  the  hypothesis  and  25%  reject  it.  Several  responses 
to  Questions  6  and  7  had  split  answers;  e.  g. ,  agree  in  magnitude,  disagree 
in  quality.  These  few  partial  agreements  were  tabulated  as  disagreements, 
but  do  not  change  the  trend  of  the  results.  The  AEDC  respondents  agree 
(12  out  of  13)  that  they  exert  little  influence  on  the  test  plan.  Thus,  it  would 
appear  that  the  facility  test  personnel  within  the  Department  of  Defense 
are  not  contributing  significantly  to  the  development  of  the  facility  test 
plan. 
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TABLE  6 

REPLIES  TO  HYPOTHESIS  THAT  DoD  TEST  FACILITY 
PERSONNEL  EXERT  VERY  LITTLE  INFLUENCE  ON 
THE  MAGNITUDE  AND  QUALITY  OF  THE 
FACILITY  TEST  PLAN 


Agency 

Disagree 

Strongly 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

0 

15 

7 

30 

6 

ASD 

0 

7 

3 

8 

0 

Headquarters 

0 

2 

1 

3 

1 

AFSC  Labs 

0 

2 

1 

7 

0 

AEDC 

0 

1 

0 

10 

2 

NASA 

0 

1 

5 

2 

0 

Other 

0 

1 

0 

2 

0 

Totals 

0 

29 

17 

62 

9 

Almost  all  military  aircraft  developments  experience  some  testing  in 


the  NASA  test  facilities.  Although  one  would  not  expect  NASA's  participa¬ 
tion  to  be  covered  in  Air  Force  Regulations  as  is  that  of  the  DoD  test  facil¬ 
ities,  it  is  likely  that  they  exert  some  direct  or  indirect  influence  on  the 
test  plan.  Question  7,  Appendix  B,  is  addressed  to  this  premise,  and 
the  results  are  noted  in  Table  7.  The  hypothesis  is  accepted  by  46%  of 
the  respondents  and  rejected  by  40%.  Comparison  of  these  results  with 
those  of  Table  6  leads  one  to  the  conclusion  that  NASA  test  personnel 
exert  more  influence  than  do  the  DoD  test  personnel  on  the  facility  test 
plans  for  development  of  aircraft. 

It  should  be  noted  that  these  results  are  not  necessarily  limited  to 
the  Air  Force  systems.  It  is  quite  possible  that  some  contractor  and  NASA 
respondents  have  also  worked  on  Navy  aircraft  development  programs  and 
have  included  these  experiences  in  their  responses.  The  study  would  be 
more  complete  if  Navy  test  personnel,  laboratories  and  development  organ¬ 
izations  had  been  surveyed,  but  it  was  necessary  to  limit  the  scope  of  the 
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study.  However,  interviews  with  respondents  indicated  that  their  respon¬ 
ses  were  primarily  directed  toward  the  Air  Force  Systems  Command,  ASD 
(as  the  developing  organization),  and  AEDC  (as  the  DoD  test  center). 


TABLE  7 

REPLIES  TO  HYPOTHESIS  THAT  NASA  TEST  FACILITY 
PERSONNEL  EXERT  VERY  LITTLE  INFLUENCE 
ON  THE  MAGNITUDE  AND  QUALITY  OF  THE 
FACILITY  TEST  PLAN 


Agency 

Disagree 

Strongly 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

3 

23 

6 

24 

2 

ASD 

0 

5 

2 

10 

1 

Headquarters 

1 

0 

1 

4 

1 

AFSC  Labs 

0 

4 

4 

2 

0 

AEDC 

1 

5 

1 

6 

0 

NASA 

2 

2 

1 

2 

1 

Other 

0 

1 

1 

1 

0 

Totals 

7 

40 

16 

49 

5 

A  New  Procedure 

It  is  appropriate  to  ask  if  the  respondent  is  satisfied  with  the  pres¬ 
ent  method  of  test  plan  origination  or  if  he  would  prefer  some  reorientation 
of  emphasis  or  procedure  to  change  the  existing  method  of  preparing  and 
evaluating  the  test  plan.  Question  8  seeks  to  determine  if  some  reorien¬ 
tation  might  be  expected  to  improve  the  present  system;  the  results  are 
shown  in  Table  8.  There  are  84%  affirmative  answers  and  12%  negative. 
Obviously  there  is  a  very  strong  belief  that  some  changes  in  test  plan 
origination  are  desirable  and  can  be  beneficial. 
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TABLE  8 

RESPONDENT'S  EXPECTATION  THAT  REORIENTATION 
OF  EMPHASIS  OR  PROCEDURE  WOULD  IMPROVE 
THE  EXISTING  METHOD  OF  PREPARING  AND 
EVALUATING  THE  FACILITY  TEST  PLAN 


Agency 

No 

Yes 

No  Answer 

Industry 

9 

45 

4 

ASD 

3 

14 

1 

Headquarters 

0 

7 

0 

AFSC  Labs 

2 

8 

0 

AEDC 

0 

13 

0 

NASA 

0 

8 

0 

Other 

0 

3 

0 

Totals 

14 

98 

5 

The  possibilities  for  changes  in  the  system  are  numerous;  however, 
it  is  desirable  to  gain  some  knowledge  of  the  experts'  preferred  direction 
of  change.  In  Question  9,  four  different  shifts  in  emphasis  and/or  pro¬ 
cedure  are  suggested  for  the  consideration  of  the  respondent;  he  is  asked 
to  compare  each  choice  with  every  other  choice  and  Indicate  his  preference. 
The  sum  of  these  preferences  is  indicative  of  the  popularity  of  the  choice. 
The  four  choices  are  repeated  here  for  the  reader's  convenience. 

a.  Continue  existing  procedures,  but  have  the  military  developing 
agency  attach  less  importance  to  the  test  plan  in  the  evaluation  of  the  bid. 

b.  Continue  existing  procedures,  but  have  the  military  developing 
agency  attach  more  importance  to  the  test  plan  in  the  evaluation  of  the  bid. 

c.  Have  the  company  bid  the  test  plan  only  as  a  lump-sum  expendi¬ 
ture  and  evolve  the  details  if  and  when  it  gets  the  contract. 

d.  Omit  the  test  plan  from  the  bid  proposal.  Develop  the  test  plan 
after  contract  award  with  the  combined  efforts  of  the  selected  contractor. 
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Systems  Project  Office,  and  government  test  facility  personnel.  Add  to  the 
contract  on  a  cost-plus-fixed-fee  basis. 

The  results  are  noted  in  Table  9.  The  numbers  in  parentheses  show 
number  of  "first  choices"  for  that  particular  suggested  change. 


TABLE  9 

RESPONDENTS'  CHOICES  FOR  CHANGES  IN  THE 
METHOD  OF  FACILITY  TEST  PLAN  ORIGINATION 


Agency 

b 

c 

d 

Industry 

50  (1) 

80  (19) 

53  (6) 

82  (19) 

ASD 

16  (1) 

37  (10) 

17  (1) 

20  (  4) 

Headquarters 

4  (0) 

13  (  3) 

12  (2) 

16  (  4) 

AFSC  Labs 

5  (0) 

19  (  5) 

9  (0) 

15  (  3) 

AEDC 

2  (0) 

24  (  3) 

9  (0) 

26  (  8) 

NASA 

4  (0) 

20  (  6) 

7  (0) 

16  (  2) 

Other 

-8..L2}. 

1  (0) 

5  (  1) 

Totals 

84  (2) 

201  (48) 

108  (9) 

180  (41) 

Not  all  respondents  who  answered  "yes"  to  Question  Number  8  made 
selections  from  the  suggested  options.  Others  had  ties  for  "first  choice,  " 
thus  resulting  in  an  indicated  100  first  choices;  48%  prefer  option  "b"  and 
41%  prefer  option  "d.  "  Industry  split  about  evenly  between  the  two  options. 
The  AEDC  respondents  were  8  to  3  in  favor  of  option  "d"  over  "b,  "  presum¬ 
ably  because  it  permits  greater  participation  on  their  part  in  preparation  of 
the  test  plan. 

As  might  be  expected,  several  alternative  plans  were  suggested  by 
the  respondents  (even  by  some  of  those  who  ranked  the  indicated  choices). 

In  the  interviews  it  became  apparent  that  a  good  many  respondents  felt 
that  they  were  forced  to  option  "b"  because  of  limited  choices  and  disagree¬ 
ment  with  some  part  of  the  other  options.  Many  respondents  did  not  want 
the  test  plan  completely  omitted  from  the  bid  proposal,  as  suggested  by 
option  "d.  "  Yet,  they  could  see  no  need  to  include  a  detailed  test  plan 
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and  agreed  that  the  test  plan  should  be  funded  on  a  "cost-plus"  basis. 
Most  of  the  industry  respondents  who  were  interviewed  liked  the  idea  of 
developing  the  final  test  plan  in  closer  concert  with  ASD  and  AEDC.  It 
was  suggested  that  NASA  personnel  should  also  be  included  if  they  are  to 
contribute  to  the  test  program.  These  respondents  indicated  that  such  a 
plan  would  tend  to  promote  harmony  among  all  parties  and  focus  attention 
to  make  the  resulting  test  program  more  successful. 

One  of  the  objections  to  a  plan  which  omits  the  detailed  test  plan 
from  the  bid  proposal  is  the  added  time  interval  after  contract  award  need¬ 
ed  to  finalize  a  facility  test  plan.  However,  most  respondents  agreed  that 
all  involved  parties  should  have  already  completed  their  "homework"  and 
that  agreement  among  the  participants  should  take  from  one  to  three  weeks 
(depending  upon  the  magnitude  of  the  test  program).  Under  such  a  plan,  it 
is  Intended  that  the  contractor  put  at  least  as  much  preparation  into  his 
test  plan  as  he  now  does;  however,  the  participating  government  organi¬ 
zations  would  be  expected  to  increase  their  present  involvement  in  defini¬ 
tion  of  the  test  plan. 

The  interviews  and  questionnaire  responses  gave  evidence  that  there 
is  some  disagreement  between  AEDC  and  ASD  personnel  as  to  the  desirable 
degree  of  AEDC  participation  in  definition  of  the  test  plan.  The  reader  will 
recall  that  by  regulation  AEDC  primarily  responds  to  requests  from  ASD. 

The  results  of  Question  6  (Table  6)  indicate  that  AEDC  personnel  are  pres¬ 
ently  contributing  little  to  test  program  planning.  It  was  noted  in  the  dis¬ 
cussion  of  Table  9  that  AEDC  prefers  the  option  that  permits  more  partici¬ 
pation  on  their  part.  It  may  also  be  observed  that  ASD  rejected  this  option 
very  strongly.  From  the  interviews  and  comments  on  the  questionnaires, 
the  author  has  concluded  that  service  funding*  at  AEDC  is  largely  respon¬ 
sible  for  the  apparent  conflict.  It  will  suffice  for  the  unfamiliar  reader  to 
know  that  this  is  a  funding  procedure  whereby  AEDC  operates  with  a  work¬ 
ing  capital  fund,  from  which  operating  expenses  are  paid,  and  which  is 
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reimbursed  through  charges  to  benefiting  organizations.  Thus,  ASD  is 
required  to  spend  its  development  funds  for  testing  at  AEDC,  and  AEDC 
is  dependent  upon  its  workload  for  its  continued  operation.  Add  to  this 
the  fact  that  NASA  is  permitted  to  perform  testing  free  for  ASD  and  one 
can  readily  see  the  basis  for  suspicion  of  motives.  AEDC  is  given  an 
incentive  to  direct  a  large  portion  of  the  test  load  to  their  own  test  facil¬ 
ities;  ASD  is  given  an  incentive  to  direct  work  away  from  AEDC  so  that 
its  development  funds  can  be  spent  for  other  purposes. 

The  author  is  not  critical  of  either  the  organizations  or  personnel; 
everyone  appears  to  be  reacting  in  a  normal  manner  to  the  incentives  pre¬ 
sented  to  him.  However,  service  funding  has  evidently  contributed  to 
tensions  and  suspicion  of  motives  among  Air  Force  organizations  that  should 
be  working  in  close  harmony.  The  evidence  collected  in  this  study  shows 
service  funding  at  AEDC  to  have  a  very  negative  influence  on  the  magni¬ 
tude  and  quality  of  the  facility  test  program  for  aerospace  systems  develop¬ 
ment. 

Recommendations 

1.  The  evidence  gathered  in  this  study  indicates  that  the  facility  testing 
program  is  compromised  at  its  inception  by  the  incentives  introduced  in 
the  competition  for  aerospace  development  contracts.  Because  the  test 
program  is  an  easily  identified  entity  and  because  the  reduction  or  elimina¬ 
tion  of  parts  of  the  test  program  incur  a  risk  that  is  not  easily  measured  or 
defined,  it  is  likely  that  the  facility  test  program  will  continue  to  be  ad¬ 
justed  unduly  to  accommodate  time  and  cost  objectives  under  the  existing 
system. 

RECOMMENDATION:  The  facility  test  plan  for  aerospace  systems  develop¬ 
ment  should  not  be  a  part  of  the  competition  for  development  contracts,  but 
should  be  added  to  the  contract  of  the  selected  contractor  on  a  cost-plus- 


fixed-fee  basis. 
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2.  The  testimony  from  the  interviews  and  questionnaires  indicates  that 
some  government  personnel  are  not  making  their  maximum  contribution  to 
the  facility  test  plan  for  systems  development.  In  particular,  it  was 
demonstrated  that  the  AFSC  test  facility  personnel  exert  little  influence 
on  the  test  plan.  Many  of  the  experienced  respondents  believe  that  the 
test  program  could  be  improved  if  all  contributors  had  the  opportunity  for 
input  and  coordination  at  the  time  of  test  plan  origination. 
RECOMMENDATION:  APR  80-14  (and/or  AFSC  Supplements)  should  be 
modified  to  require  more  participation  and  coordination  by  the  involved 
government  agencies  in  development  of  the  facility  test  plan.  The  in¬ 
volvement  of  the  AFSC  test  facility  personnel  should  be  standard  proce¬ 
dure;  i.  e. ,  not  so  dependent  upon  requests  from  the  developing  agency. 

3.  There  is  reason  to  believe  that  service  funding  at  AEDC  is  producing 
incentives  which  are  hindering  coordination  among  government  agencies 
and  which  are  reducing  the  effectiveness  of  the  facility  test  program. 
RECOMMENDATION:  Service  funding  at  test  centers  should  be  subjected 
to  a  critical  review  at  AFSC  and  USAF.  Detrimental  effects  of  this  pro¬ 
cedure  should  be  carefully  weighed  against  the  claimed  benefits.  (This 
subject  is  further  discussed  in  a  later  chapter.) 


CHAPTER  IV 


EMPHASIS  IN  THE  SYSTEMS  DEVELOPMENT  CYCLE 

The  concerns  of  the  officials  of  the  Department  of  Defense  and  their 
recommended  changes  in  the  military  system  development  and  acquisition 
cycle  were  mentioned  in  Chapter  I.  On  May  28,  1970,  Deputy  Secretary 
of  Defense  Packard  sent  a  memorandum  to  the  DoD  hierarchy  and  military 
departments  in  which  he  outlined  broad  policy  guidance  for  the  acquisition 
and  development  of  major  weapon  systems  (7).  In  this  memorandum,  the 
Deputy  Secretary  covered  management,  development  (conceptual  and  full- 
scale),  production,  and  contracts.  It  was  intended  that  the  broad  policy 
guidance  be  translated  into  appropriate  action  by  all  the  Mlitary  Services. 

The  memorandum  has  drawn  widespread  acclaim  from  most  of  those 
affected  and  the  proposed  shift  in  development  policy  has  been  praised  in 
many  of  the  publications  of  the  aerospace  industry.  Dr.  Karl  Harr,  Jr. , 
president  of  the  Aerospace  Industries  Association,  saw  the  memo  as  "one 
of  the  most  potentially  constructive  documents  generated  within  the  Pen¬ 
tagon  in  years"  (8).  All  respondents  to  the  questionnaires  had  heard  of 
the  new  policy  and  were  aware  that  some  changes  in  the  aerospace  develop¬ 
ment  and  acquisition  cycle  were  expected.  However,  there  were  some  dif¬ 
ferences  in  interpretation  regarding  the  proposed  changes,  perhaps  because 
the  policy  guidance  is  so  inclusive. 

Some  elements  of  the  new  policy  guidance  are  not  so  elusive  and 
are  being  implemented.  In  the  past,  certain  aircraft  development  programs 
have  deliberately  introduced  tooling  for  production — and  sometimes  initiated 
production — before  it  was  demonstrated  that  the  aircraft  would  perform  sat¬ 
isfactorily  (e.  g. ,  the  F-111).  The  policy  guidance  made  it  very  clear  that 
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this  was  one  procedure  that  should  be  corrected,  for  changes  in  tooling 
are  costly  and  modifications  to  unsatisfactory  aircraft  are  usually  even 
more  costly.  Corrective  action  was  clear;  one  should  fly  the  aircraft  and 
demonstrate  its  performance  before  making  a  production  decision.  The 
penalty  is  a  longer  development  cycle;  the  benefits  are  obvious.  Some 
of  the  other  changes  directed  by  the  Deputy  Secretary  regarding  type  of 
development  contracts  and  certain  management  practices  have  also  been 
interpreted  and  implemented  to  some  degree.  However,  the  author  has 
observed  that  procedures  proposed  for  implementation  of  the  directives  do 
not  appear  to  fully  recognize  the  possibilities  for  "risk  identification  and 
reduction,  evaluation  of  technical  achievements  and  demonstration  of 
performance"  prior  to  flight  of  the  aircraft.  Yet  the  policy  guidance  is 
worded  so  that  it  seems  undeniable  that  such  emphasis  is  expected. 

The  hypotheses  to  be  addressed  in  this  chapter  are  centered  around 
the  proposition  that  risk  Identification  and  reduction,  technical  evalua¬ 
tion  and  hardware  demonstration  are  important  in  the  aircraft  development 
cycle  prior  to  flight  of  the  aircraft;  it  is  suggested  that  these  functions 
are  not  receiving  sufficient  attention  and  emphasis.  The  role  of  the  test 
facility  in  support  of  this  proposition  is  explored  in  different  settings; 
first  in  relation  to  the  new  DoD  development  policy,  and  then  as  a  contrib¬ 
utor  to  the  management  decision-making  process.  It  is  then  hypothesized 
that  existing  procedures  for  processing  information  for  the  development 
decisions  are  not  entirely  adequate.  As  before,  the  author  has  turned  to 
the  practicing  aerospace  experts  to  test  the  hypotheses  and  to  solicit  their 
advice  regarding  the  proper  role  of  the  test  facility.  Those  who  are  not 
familiar  with  the  function  of  the  test  facility  in  support  of  system  develop¬ 
ment  are  directed  to  a  thesis  by  Col.  Joseph  Henry  at  the  Air  War  College 
(9).  Col.  Henry  has  presented  a  convincing  argument  for  use  of  test  facil¬ 
ities  in  implementation  of  parts  of  the  new  development  philosophy,  and 
has  illustrated  his  position  with  historical  examples  drawn  primarily  from 
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propulsion  system  development. 

The  reader  should  be  aware  that  the  term  "wind  tunnel  testing,  " 
although  widely  used  in  industry,  is  usually  a  misnomer.  "Testing"  im¬ 
plies  that  one  is  seeking  to  determine  whether  or  not  a  design  is  adequate, 
whereas  most  wind  tunnel  "tests"  are  conducted  to  gather  basic  design 
data,  either  to  evolve  an  adequate  configuration  or  to  gather  definitive 
design  data  on  a  configuration  previously  determined  to  be  satisfactory. 
Hence,  most  wind  tunnel  "tests"  are  not  tests  at  all,  but  are  aerodynamic 
simulation  studies  (10).  This  differentiation  in  test  emphasis  may  be 
observed  in  the  later  discussions. 

Agreement  with  the  New  Development  Philosophy 

It  is  first  desirable  to  determine  the  strength  of  agreement  of  the 
aerospace  experts  with  portions  of  the  philosophy  of  systems  development 
as  interpreted  by  the  author.  Question  10,  Appendix  B,  is  designed  to 
ascertain  the  respondents'  agreement  with  emphasis  on  risk  reduction,  tech¬ 
nical  evaluation,  correction  of  mistakes  and  appropriate  trade-offs  early  in 
the  development  cycle  before  completion  of  the  flight  vehicle.  (The  reader 
may  wish  to  review  the  preamble  to  Questions  10-14  in  Appendix  B  so  that 
he  will  have  a  better  feeling  for  the  intent  of  the  author  and  respondents.) 
The  results  of  this  question  are  shown  in  Table  10.  There  is  obviously 
overwhelming  acceptance  of  the  new  policy  and  pre-flight  emphasis;  how¬ 
ever,  from  the  respondents'  comments  on  the  questionnaires  and  in  the 
interviews,  it  was  evident  that  a  few  respondents  who  answered  in  the 
affirmative  did  not  agree  with  the  proposition  as  posed  by  the  author,  but 
instead  have  agreed  with  other  portions  of  the  policy  as  they  have  chosen 
to  interpret  them.  Also,  there  were  many  qualified  agreements  and  reserva¬ 
tions  explained  in  the  comments  to  this  question.  In  the  following  para¬ 
graphs,  the  author  will  attempt  to  group  some  of  these  comments  and  explain 
the  various  positions. 
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TABLE  10 

RESPONDENTS’  AGREEMENT  WITH  EMPHASIS  IN 
THE  NEW  DEVELOPMENT  PHILOSOPHY  ON 
PRE-FLIGHT  EVALUATION  DEMONSTRATION 
AND  RISK  REDUCTION 


Agency 

Strongly 

Disagree 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

0 

3 

2 

35 

18 

ASD 

0 

0 

0 

9 

9 

Headquarters 

0 

0 

0 

3 

4 

AFSC  Lab 

0 

1 

0 

4 

5 

AEDC 

0 

0 

0 

5 

8 

NASA 

0 

0 

0 

2 

6 

Other 

0 

0 

0 

3 

0 

Totals 

0 

4 

2 

61 

50 

Most  respondents  interpret  the  new  philosophy  as  a  peace-time 
development  approach,  where  cost  savings  are  more  important  than  de¬ 
velopment  time.  This  "step-by-step"  development  cycle  is  expected  to 
assure  (within  reason)  that  problems  have  been  solved  before  more  sub¬ 
stantial  Investment  is  made  in  the  advanced  stages  of  development.  This 
is  opposed  to  the  war-time  philosophy,  where  it  is  necessary  to  take 
chances  with  concurrent  development  efforts  just  to  get  the  new  system 
into  the  national  arsenal  at  an  early  date.  Most  respondents  agree  that 
the  new  emphasis  in  development  will  likely  extend  development  time; 
however,  only  a  few  see  this  as  having  a  detrimental  effect.  Most  respond¬ 
ents  expect  overall  development  costs  to  decrease.  Only  a  few  of  the 
respondents  are  making  a  direct  association  between  the  expenditure  of 
time  and  money.  Such  an  association  was  certainly  accurate  for  the 
accelerated  concurrency  approach,  where  any  delay  in  the  development 
program  has  usually  been  very  expensive  because  of  continuing  but  unpro¬ 
ductive  concurrent  phases.  However,  most  respondents  expect  the  new 
development  cycle  to  account  for  adequate  time  in  each  step  of  development, 
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with  build-up  in  the  next  phase  curtailed  until  the  appropriate  time. 

There  are  some  notes  of  caution  which  practitioners  of  the  new 
policy  might  well  observe.  Some  respondents  fear  that  the  philosophy 
may  not  permit  rapid  response  to  a  threat  from  a  potential  enemy.  The 
author  does  not  share  this  concern;  if  the  technology  base  is  maintained, 
it  should  be  relatively  easy  to  accelerate  the  development  cycle  to  meet 
a  threat.  A  more  important  consideration  may  be  the  hesitancy  to  under¬ 
take  a  high-risk  development  project  which  offers  a  very  good  payoff. 

Also,  some  respondents  caution  that  a  program  can  be  drawn  out  time-wise 
to  the  point  where  it  loses  its  glamour  and  its  support  withers;  e.  g.  ,  the 
Supersonic  Transport  (SST)  and  DYNASOAR. 

Several  respondents  indicated  agreement  with  the  policy,  but  suggest 
that  its  applicability  depends  upon  the  nature  of  the  system  being  developed; 
i.  e. ,  conventional  or  advanced.  They  argue  that  the  policy  implementa¬ 
tion  would  improve  a  development  program  with  a  great  deal  of  advanced 
technology,  but  would  only  hinder  a  more  conventional  development  by 
stretch-out.  A  few  suggest  that  only  certain  elements  of  the  system  be 
included  in  this  concept  (the  airframe,  propulsion  system,  flight  controls 
and  fly-away  components),  with  other  elements  developed  via  a  different 
approach  (avionics,  fire  control,  etc.). 

Most  respondents  support  the  new  policy  without  qualification.  They 
say  that  implementation  of  this  policy  will  result  in  better  systems  and 
eliminate  bad  systems  at  less  cost.  They  believe  that  it  can  be  accom¬ 
plished  just  as  quickly  and  more  cheaply  if  the  development  program  is 
planned  this  way  from  the  beginning  and  executed  according  to  plan.  It 
is  argued  that  it  is  far  easier  and  less  costly  to  determine  and  correct  de¬ 
ficiencies  early  in  the  development  program  rather  than  to  engage  in  modi¬ 
fications  to  flight  vehicles — even  the  initial  prototype  flight  vehicle. 

Many  of  the  respondents  who  readily  accept  the  new  development 
policy  and  emphasis  have  revealed  a  great  deal  of  skepticism  that  it  will 
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be  Implemented.  The  argument  here  does  not  address  prototype  flight 
demonstration  prior  to  a  production  decision;  all  respondents  seem  to 
agree  that  this  is  desirable  and  will  be  practiced  because  the  implemen¬ 
tation  is  so  obvious.  Instead,  the  skepticism  centers  around  recognition 
and  practice  of  techniques  to  gather  design  data,  reduce  risk,  and  demon¬ 
strate  performance  prior  to  vehicle  flight.  Two  of  the  comments  from  re¬ 
spected  members  of  industry  sum  the  contractors'  majority  viewpoint  very 
well: 

"Until  the  government  is  willing  to  honestly  face  the  early  cost 
impact  of  doing  business  this  way,  and  is  willing  to  give  an 
honest  contractor  credit  in  his  proposal  for  recognizing  and 
addressing  this  problem  cost-wise,  there  is  little  likelihood 
for  improvement.  " 

"The  theory  is  excellent;  but  in  the  real  world  with  limited 
development  funds,  even  this  approach  tends  to  de-emphasize  g 
the  importance  of  testing;  e.  g. ,  the  B-1  development  program.  " 

Test  Facility  Role  in  the  New  Policy 

It  was  demonstrated  in  the  preceding  discussion  that  the  proposed 
emphasis  on  pre-flight  aircraft  development  was  accepted  by  almost  all 
the  aerospace  experts.  In  this  section  of  the  report,  the  role  of  the  test 
facility  in  implementation  of  this  policy  emphasis  will  be  examined.  It 
is  hypothesized  that  the  shift  in  emphasis  in  the  development  cycle  will 
permit  the  test  facility  to  be  more  useful  and  influence  the  system  develop¬ 
ment  more  strongly.  This  proposition  was  posed  in  Question  11,  Appendix 
B,  and  the  results  are  shown  in  Table  11. 


8 

The  B-1  Advanced  Strategic  Bomber  is  being  developed  by  the 
North  American  Rockwell  Corporation. 
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TABLE  11 

REPLIES  TO  HYPOTHESIS  THAT  THE  SHIFT  IN  EMPHASIS 
IN  THE  DEVELOPMENT  CYCLE  WILL  PERMIT  THE 
TEST  FACILITY  TO  BE  MORE  USEFUL  AND 
INFLUENTIAL  IN  SYSTEMS'  DEVELOPMENT 


Agency 

Strongly 

Disagree 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

1 

10 

3 

37 

7 

ASD 

0 

2 

2 

13 

1 

Headquarters 

0 

0 

1 

4 

2 

AFSC  Labs 

0 

0 

0 

8 

2 

AEDC 

2 

0 

2 

5 

4 

NASA 

0 

1 

0 

5 

2 

Other 

0 

1 

0 

2 

0 

Totals 

3 

14 

8 

74 

18 

Although  these  results  substantiate  the  hypothesis  by  a  large  major¬ 
ity  (79%  agree,  14%  disagree),  they  do  not  fully  indicate  the  support  given 
the  hypothesis.  The  wording  of  the  question  made  the  answer  uncertain 
for  the  respondent  who  was  skeptical  about  implementation  of  the  policy. 
Many  of  the  respondents  reworded  the  question  before  answering  to  indi¬ 
cate  that  the  test  facility  "could"  or  "should"  be  more  useful;  however, 
they  were  reluctant  to  state  that  it  "would"  be.  Over  one-half  of  those 
who  disagree  or  disagree  strongly  with  the  hypothesis,  stated  in  their  com¬ 
ments  that  their  disagreement  was  based  on  the  same  interpretation.  Actual¬ 
ly,  only  about  5%  of  the  respondents  disagree  with  the  proposition  intended 
in  the  question.  These  5%  appear  to  believe  that  the  prototype  aircraft 
should  be  built  and  put  into  the  air  as  quickly  as  possible.  They  see  the 
test  facility  in  the  primary  role  of  complementing  flight  evaluation  and  cor¬ 
recting  deficiencies  found  in  flight.  Following  are  some  of  the  comments 
which  seem  to  best  represent  the  attitude  of  the  95%.  These  comments 
have  been  selected  from  those  in  industry  who  use  the  data  to  design  air¬ 
craft.  The  reader  will  observe  in  these  comments  a  confidence  in  the  use 
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of  the  test  facility,  a  belief  that  the  policy  could  be  implemented,  but  a 

reluctance  to  believe  that  it  will. 

"Technical  risk  can  be  assessed  prior  to  flight — very  accurately. 

The  Military  Developing  Agency  schedules  do  not  allow  configura¬ 
tion  development  and  final  production  version  verification  through 
wind  tunnels,  simulator  and  static  test  to  occur  prior  to  design 
and  tooling  go-ahead.  If  it  did,  untold  billions  ($)  in  cost  would 
be  saved — at  a  cost  of  about  a  year  in  the  development  schedule.  " 

"By  now  we  should  be  able  to  develop  in  ground-based  facilities 
(wind  tunnel/simulators)  an  aircraft  configuration  without  flying 
the  hardware.  " 

"The  new  policy  will  tend  to  further  compress  test  facility  time. 
Management  will  feel  that  risk  is  less;  i.  e. ,  they  will  feel  that 
they  can  'fix  it  in  flight.  '  Too  few  decision-makers  understand 
that  relatively  risk-free  preflight  configuration  development  is 
possible.  " 

"The  policy  is  being  interpreted  to  place  more  emphasis  on  the 
flight  vehicle.  This  will  tend  to  shorten  the  preflight  develop¬ 
ment  cycle;  thus,  test  results  will  not  get  into  the  design  and 
risk  will  be  increased.  " 

"The  time  scale  for  development  testing  in  facilities  is  com¬ 
pressed.  There  may  be  even  more  reliance  on  flight  testing  than 
in  the  past.  " 

As  a  complement  to  the  hypothesis  of  Question  11  (Table  11),  it  is 
desirable  to  determine  how  the  aerospace  experts  expect  the  magnitude  of 
facility  testing  to  be  influenced  by  the  shift  in  development  emphasis.  The 
respondent  is  asked  in  Question  12,  Appendix  B,  if  this  new  development 
philosophy  will  require  more  or  less  facility  testing;  the  results  are  shown 
in  Table  12. 

Obviously,  the  respondents  do  not  expect  the  output  of  the  test  facil¬ 
ity  to  be  reduced  as  a  result  of  the  new  policy.  Most  of  the  respondents 
interpreted  the  question  as  it  was  intended,  i.  e. ,  more  or  less  testing  on 
each  individual  development  project.  At  least  two  of  those  who  responded 
"less"  did  so  because  they  expect  fewer  system  development  programs  and 
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and  therefore  less  overall  facility  testing.  Another  two  of  the  six  who 
replied  "less"  expect  less  testing  on  each  development  project  in  the  long- 
run  because  of  decreased  testing  in  the  facility  to  correct  deficiencies 
discovered  in  flight.  At  least  one  respondent  believes  that  there  will  be 
more  testing  because  there  will  be  more  time  for  testing  and  people  will 
find  things  to  test.  As  in  the  preceding  question,  several  respondents 
reworded  the  question  to  read  "should  require"  rather  than  "will  require.  " 

TABLE  12 

RESPONDENTS'  ESTIMATES  OF  THE  EFFECT  OF  THE 
NEW  DEVELOPMENT  POLICY  ON  TEST 
FACILITY  WORKLOAD 


Agency 

Less 

No  Change 

More 

No  Answer 

Industry 

3 

23 

29 

3 

ASD 

0 

7 

11 

0 

Headquarters 

0 

1 

5 

1 

AFSC  Labs 

1 

3 

6 

0 

AEDC 

1 

2 

10 

0 

NASA 

1 

3 

3 

1 

Other 

0 

1 

1 

1 

6 

40 

65 

6 

The  responses  shown  in  Tables  11  and  12  clearly  demonstrate  the 
respondents'  belief  that  the  test  facility  can  and  should  play  a  more 
influential  role  in  aerospace  systems  development;  implementation  of 
this  philosophy  is  expected  to  increase  the  use  of  the  test  facility  in 
support  of  each  system  development  program. 

Confirmation  Testing  in  the  Facility 

DoD  has  adjusted  functional  responsibilities  for  acquiring  major  weap¬ 
on  systems  so  that  decision  points  arise  at  the  beginning  of  several  critical 
phases  (3).  Figure  1  shows  an  abbreviated  development  cycle  with  the 
three  decision  points  introduced.  Note  that  the  decisions  are  reserved  for 
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DoD  and  that  two  of  the  three  decisions  must  be  made  prior  to  full-scale 
development.  Although  the  Validation  Phase  may  include  a  prototype 
flight  vehicle  (or  even  competitive  prototypes),  the  more  usual  case  will 
be  first  flight  in  the  Full-Scale  Development  Phase.  The  high  costs  asso¬ 
ciated  with  prototype  development  will  usually  prevent  such  a  philosophy 
from  being  pursued  in  the  Validation  Phase  on  both  advanced  design  and 
large,  expensive  aircraft.  Since  the  emphasis  in  the  policy  guidance  is 
on  demonstration  of  performance  and  progress  with  hardware  and  reduction 
of  reliance  on  paper  studies,  it  is  suggested  that  the  test  facility  can  pro¬ 
vide  information  at  these  decision  points  of  benefit  to  the  decision-maker. 

In  the  interim  between  paper  studies  and  flight  demonstrations,  the  author 
is  unaware  of  any  other  instruments  to  measure  technical  progress  except 
test  facilities,  flight  simulators,  etc.  This  proposition  is  addressed  in 
Question  13,  Appendix  B,  and  the  respondent  is  asked  what  contributions 
the  facility  might  make  in  an  evaluation  role  at  these  major  decision  points 
prior  to  aircraft  flight.  The  intent  is  to  define  the  type  of  hardware  demon¬ 
strations  and  milestone  checks  in  each  phase  that  will  provide  the  confi¬ 
dence  needed  by  the  decision-maker.  Some  of  the  statements  made  by  the 
respondents  are  quoted  in  Exhibits  1  and  2,  Appendix  D.  Although  both  the 
Conceptual  Phase  and  Validation  Phase  are  briefly  described  in  Question  13, 
some  supplementary  comments  seem  appropriate  prior  to  a  discussion  of  the 
respondents'  replies.  The  reader  should  be  aware  that  the  objective  here 
is  not  to  cover  the  complete  scope  of  each  development  phase,  but  only  that 
portion  which  might  be  supported  by  technical  data  from  test  facilities. 

The  Conceptual  Phase  has  as  its  objective  the  definition  and  selection 

9 

of  systems  which  warrant  further  development.  Economic,  political  and 


The  descriptive  comments  on  the  Conceptual  and  Validation  Phases 
are  taken  from  a  Draft  Interim  AFSC  Pamphlet  entitled,  "Systems  Manage¬ 
ment,  Guide  for  Management  on  the  Systems  Acquisition  Life  Cycle.  " 
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technical  trade-offs  are  performed  and  tested  against  military  need.  Tech¬ 
nical  uncertainties  are  supposed  to  be  Identified  in  the  Conceptual  Phase 
for  resolution  during  the  Validation  Phase.  However,  it  is  likely  that 
some  minimum  state-of-the  art  demonstration  will  be  required  for  key  sub¬ 
systems  and  components.  Preliminary  designs  are  evaluated  from  the 
standpoint  of  feasibility  and  technical  risk.  In  the  first  part  of  Question 
13,  the  respondents  are  asked  to  indicate  the  kind  of  information  which 
decision-makers  should  have  available  from  the  test  facility  to  support 
their  Program  Decision.  Some  of  the  representative  comments  are  quoted 
in  Exhibit  1,  Appendix  D. 

The  Validation  Phase  has  as  its  objective  the  establishment  of  firm 
and  realistic  performance  specifications  which  meet  operational  require¬ 
ments.  Major  program  characteristics  (cost,  schedule,  and  technical)  are 
validated  and  refined  through  extensive  study,  analysis,  hardware  develop¬ 
ment  and  sometimes  prototype  testing.  This  Phase  should  provide  high 
confidence  that  risks  have  been  resolved  (or  minimized)  and  that  the  prob¬ 
ability  of  successful  development  is  great.  The  decision-maker  at  the  DoD 
will  use  this  information  to  decide  whether  or  not  to  proceed  into  the  Full- 
Scale  Development  Phase.  The  initial  Category  I  test  plan  is  prepared  dur¬ 
ing  the  Validation  Phase,  and  certain  parts  of  the  plan  are  executed.  Al¬ 
though  the  tests  are  conducted  primarily  by  the  contractor,  the  Air  Force 
does  participate  in,  evaluate  and  control  the  testing  through  the  Systems 
Project  Office.  By  the  end  of  this  Phase  the  "anticipated  unknowns"  should 
be  isolated,  described  in  quantitative  terms,  and  essentially  eliminated  as 
problem  areas.  The  "unanticipated  unknowns"  should  be  determined  as 
well  as  possible,  but  many  hardware  interface  problems  must  necessarily 
await  actual  evaluation  in  the  real  environment.  Exhibit  2,  Appendix  D, 
provides  typical  comments  from  the  respondents  on  the  type  of  data  and 
information  which  the  test  facility  should  and  could  provide  to  the  decision¬ 
maker  for  his  Ratification  Decision. 
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All  of  the  respondents  see  some  useful  role  for  the  test,  facility 
during  the  Conceptual  Phase.  The  comments  of  the  experts  are  centered 
around  the  use  of  the  facility  for  technology  demonstrations,  trade-off 
studies,  feasibility  studies,  configuration  evaluations,  uncertainty  and 
risk  identification  and  preliminary  design  information.  Only  four  of  the 
117  respondents  were  negative  with  regard  to  the  extended  use  of  the 
facility  during  the  Validation  Phase,  and  most  respondents  are  of  the  opin¬ 
ion  that  the  facility  has  its  greatest  pay-off  during  this  phase.  Three  of 
the  objections  were  concerned  with  the  use  of  prototypes  during  this  phase. 
These  respondents  felt  that  the  facility  contribution  should  primarily  be 
confined  to  the  Conceptual  Phase,  and  effort  here  devoted  to  flight  proto¬ 
types.  Another  respondent  questioned  the  accuracy  and  validity  of  the 
facility  test  data  to  such  an  extent  that  he,  too,  advocated  only  prototype 
flight  during  this  phase.  However,  the  comments  representative  of  most 
respondents  in  Exhibits  1  and  2,  Appendix  D,  illustrate  that  these  exper¬ 
ienced  aerospace  personnel  see  a  critical  and  necessary  role  for  the  test 
facility  in  support  of  management  decisions.  It  is  accepted  that  the  test 
facility  can  be  more  than  just  the  engineer's  tool  to  provide  research  and 
design  information;  it  can  also  be  an  effective  means  of  measurement  and 
evaluation  and  thus  support  the  decision-making  process.  This  hypothesis 
is  supported  to  some  extent  in  an  AIAA  Paper  by  Messrs.  Wagaman  and 
Yeager  of  the  Mitre  Corporation  (11). 

A  few  respondents  made  the  observation  that  "the  A-X  is  not  the  way 
to  do  it.  "  The  Air  Force  is  presently  developing  the  A-X  specialized  close 
air  support  aircraft  via  parallel  development  programs;  both  contractors 
will  build  a  prototype  for  flight  competition.  Two  arguments  were  presented. 
One  held  that  the  government  lost  too  much  control  with  a  "hands-off"  policy 
during  the  development  of  the  prototype.  It  was  argued  that  the  government 
had  too  much  invested  and  too  much  at  stake  to  permit  such  loose  controls. 
The  other  argument  was  that  too  much  dependence  is  being  placed  on  the 
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competition  and  flight  tests  to  produce  good  results. 

The  author  has  discussed  this  parallel  development  concept  with 
members  of  industry  and  can  report  good  agreement  on  the  following  points. 
The  competition  between  two  companies  sponsors  extra  effort;  both  com¬ 
panies  are  encouraged  to  place  their  very  best  personnel  on  the  job  and 
give  the  development  program  highest  priority  within  the  company.  Minimum 
interference  by  the  government  monitors  during  development  permits  much 
better  planning,  implementation  of  plans,  and  use  of  technical  personnel  by 
the  contractor.  One  contractor  estimated  that  he  could  get  30%  more  tech¬ 
nical  effort  out  of  his  engineering  force  since  they  did  not  have  to  answer 
so  many  questions  and  complete  so  much  documentation.  On  the  other 
hand,  most  respondents  agree  that  development  funds  are  probably  always 
going  to  be  limited.  There  will  likely  not  be  enough  money  for  two  good 
development  programs.  Consequently,  the  competitive  programs  will  be 
restricted  in  money,  time,  and  use  of  the  test  facility  (like  the  A-50.  His¬ 
tory  has  demonstrated  that  this  has  been  the  type  of  development  environ¬ 
ment  that  runs  a  high  risk  of  producing  unsatisfactory  flight  aircraft.  It 
would  appear  that  the  benefits  of  competition  do  not  compensate  for  lack 
of  design  data;  two  parallel  but  restricted  development  programs  are  not 
as  likely  to  produce  a  good  aircraft  as  one  thorough  development  program. 
Those  who  commented  were  hopeful  that  the  A-X  program  would  not  befall 
these  ills  since  it  is  a  relatively  straightforward  and  conventional  design. 

Some  respondents  made  a  recommendation  for  competition  between 
scaled  aircraft  models  (model  fly-offs)  in  the  wind  tunnel  during  the  Valida¬ 
tion  Phase.  The  author  has  pursued  and  developed  this  idea  in  interview 
with  Industry  personnel  and  found  very  good  acceptance.  This  approach 
brings  together  many  of  the  suggestions  made  by  the  respondents,  pro¬ 
vides  a  competitive  atmosphere,  yet  does  not  incur  the  expense  of  the  com¬ 
petitive  flight  prototypes.  It  is  suggested  that  the  competing  contractors 
provide  an  aerodynamic  model  (as  per  specifications)  to  the  government  as 
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part  of  their  proposal.  The  model  should  represent  the  contractor's  best 
estimate  of  his  proposed  final  design.  The  government  personnel  would 
evaluate  the  competing  models  in  the  best  government  facilities  available 
and  thus  assure  that  facility  data  and  corrections  were  common.  The  test 
results  would  become  a  part  of  the  contractor's  proposal  and  be  included 
with  the  remainder  of  the  evaluations.  The  contractors  seemed  to  feel 
that  this  procedure  is  fair  and  provides  evidence  to  permit  selection  of  the 
best  aircraft.  This  is  evidently  a  compromise  proposal;  the  assumption  is 
that  development  funds  will  not  permit  competitive  prototype  design  and  fly- 
off  of  advanced  or  large  aircraft,  and  will  not  permit  competitive  and  thor¬ 
ough  development  programs  for  conventional  aircraft.  At  present,  it  is 
extremely  difficult  for  the  government  evaluators  to  determine  the  best 
"paper"  aircraft  when  each  potential  contractor  presents  data  which  has 
been  accumulated  from  different  (and  sometimes  inadequate)  test  facilities 
and  subjected  to  various  corrections. 

It  should  be  noted  that  the  idea  of  competitive  wind  tunnel  fly-offs 
has  already  been  tested  to  a  limited  extent  by  the  Air  Force  and  is  report¬ 
edly  planned  by  NASA  for  the  Space  Shuttle  Project.  Although  some  success 
may  be  claimed,  better  control  of  the  variables  as  suggested  by  the  respond¬ 
ents  should  improve  the  usefulness  and  effectiveness  of  the  method.  It  was 
mentioned  by  one  respondent  that  competitive  fly-offs  might  also  be  appli¬ 
cable  for  the  Conceptual  Phase  for  differentiating  among  concepts  and 
ascertaining  impartial  estimates  of  risk  and  probability  of  meeting  opera¬ 
tional  requirements.  The  author  agrees  that  extension  of  this  philosophy 
to  the  Conceptual  Phase  may  very  well  be  worthwhile  for  an  expensive  and/ 
or  technically  complicated  development  program. 

Management  of  the  Evaluation  Function 

The  test  facility  already  plays  a  substantial  role  in  aircraft  develop¬ 
ment,  and  in  the  future  may  play  an  even  more  important  role.  Since  data 
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from  the  test  facility  can  be  used  to  measure  progress  and  can  influence 
program  continuation,  it  behooves  the  Air  Force  to  give  some  attention  to 
the  process  by  which  validation  information  from  the  test  facility  is  inter¬ 
preted,  evaluated  and  passed  to  the  decision-maker.  It  is  suggested  that 
existing  procedures  within  AFSC  do  not  always  permit  the  processing  of 
such  validation  information  with  the  necessary  objectivity  and  freedom  from 
bias. 

The  test  and  evaluation  functions  have  recently  been  addressed  by  a 
distinguished  panel  in  a  study  of  the  DoD  and  its  operating  procedures  for 
the  Secretary  of  Defense  and  the  President.  The  report  on  this  study  is 
commonly  called  the  "Blue  Ribbon  Report"  or  "Fitzhugh  Report"  (named  after 
the  chairman,  Gilbert  W.  Fitzhugh).  The  following  is  quoted  from  this 
document: 

In  connection  with  test  and  evaluation,  it  should  be  emphasized  that 
responsibilities  for  any  evaluation  function  must  be  exercised  inde¬ 
pendently.  When  they  are  subjected  to  or  combined  with  responsi¬ 
bilities  for  the  development  of  the  item  or  subject  being  evaluated, 
the  requisite  objectivity  is  seriously  jeopardized.  (12) 

In  a  report  from  the  Air  Force  Propulsion  Laboratory,  the  following  recom¬ 
mendation  was  made  with  regard  to  aircraft  engine  testing  procedures: 

The  Test  Center  must  not  be  subjected  to  influence  or  pressure  from 
the  system's  contractual  biases  relative  to  the  determination  of 
contractual  performance  points.  (13) 

These  remarks  demonstrate  the  awareness  on  the  part  of  knowledgeable 
observers  that  the  evaluation  of  the  aerospace  system's  development 
progress  may  be  subject  to  prejudicial  influence  by  those  interested  in 
demonstrating  development  objectives. 

In  Question  14,  Appendix  B,  the  respondent  is  introduced  to  the  problem 
of  getting  unbiased  information  to  the  decision-maker.  He  is  particularly 
asked  to  suggest  an  appropriate  procedure  for  analyzing  and  reporting  on 
experimental  data  from  the  wind  tunnel  that  is  to  be  used  in  ascer¬ 
taining  compliance  with  expected  performance  (as  suggested  in  the 
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responses  to  Question  13).  At  least  three  parties  are  to  be  included  in 
the  discussion;  i.  e. ,  the  contractor.  Systems  Project  Office,  and  DoD 
Test  Center.  Some  of  the  responses  are  quoted  in  Exhibit  3,  Appendix  D. 
The  author  has  attempted  to  include  replies  which  are  representative  of 
many  points  of  view.  A  large  number  of  responses  have  been  reproduced 
to  give  the  reader  some  knowledge  of  the  breadth  of  opinion  and  depth  of 
feeling  on  this  subject. 

A  very  few  of  the  respondents  have  denied  existence  of  the  problem; 
these  few  were  quite  defensive  and  argued  that  professionalism  and  indi¬ 
vidual  integrity  would  not  permit  such  biased  information.  However,  most 
of  the  respondents  did  acknowledge  the  existence  of  the  problem;  they 
claimed  that  optimistic  interpretation  of  validation  information  may  always 
be  expected  from  those  who  are  trying  to  "sell"  a  particular  program  or 
who  have  a  very  close  and  vested  interest  in  the  program.  Some  respond¬ 
ents  noted  examples  from  their  own  personal  experiences;  however,  these 
have  been  omitted  to  preserve  respondent  anonymity. 

It  is  impossible  to  summarize  the  many  individual  ideas  given  in 
response  to  Question  14.  The  suggestions  usually  involve  several  thoughts 
or  actions,  and  an  attempt  to  group  them  into  representative  categories  has 
proven  unrewarding.  The  author  has  chosen,  instead,  to  analyze  the  ob¬ 
jectivity  of  the  participants  in  the  evaluation  process  and  to  point  out 
certain  ideas  that  are  contained  in  many  of  the  proposals.  Some  of  these 
thoughts  will  serve  as  the  basis  for  recommendations  to  be  proposed  later 
in  this  chapter. 

It  is  not  intended  to  imply  that  any  group  has  more  or  less  honesty 
and  integrity  than  any  other.  One  would  only  expect  that  the  involved  and 
interested  party  would  make  optimistic  Interpretations  of  data  if  that  meant 
conformity  with  specified  performance  levels,  especially  if  program  con¬ 
tinuation  depended  upon  such  interpretations.  It  must  be  acknowledged 
that  data  from  the  test  facility  are  sometimes  subject  to  a  wide  range  of 
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interpretation,  depending  primarily  upon  corrections  which  may  be  applied 
to  the  data.  The  contractor's  role  and  incentives  need  no  explanation.  He 
has  much  to  lose  if  a  program  is  cancelled  prior  to  the  Production  Phase. 
The  SPD  usually  finds  himself  in  a  position  where  he  is  trying  to  maintain 
a  development  program  with  time  and  money  limitations  which  permit  only 
success  at  each  step  of  the  development  process.  The  pressure  to  meet 
predetermined  objectives  and  maintain  schedule  is  enormous;  the  alter¬ 
native  is  to  ask  for  more  time  and/or  money.  This  is  too  often  interpreted 
as  "bad  management. "  The  government  test  facility  personnel  are  usually 
thought  to  have  no  direct  pressures  which  would  jeopardize  their  objec¬ 
tivity.  However,  the  pressures  brought  about  by  operation  of  the  Test 
Centers  under  service  funding  have  modified  this  position  somewhat.  Can¬ 
cellation  of  a  development  project  can  result  in  closed  facilities  and  loss 
of  jobs.  It  seems  then  that  all  parties  involved  in  the  evaluation  process 
have  some  "vested  interests"  which  could  reduce  their  objectivity. 

Several  suggestions  were  made  by  the  respondents  which  included 
the  use  of  "outside  observers"  for  interpretation  and  evaluation.  Unfor¬ 
tunately,  uninvolved  individuals  and  groups  often  do  not  have  the  depth 
of  detailed  knowledge  to  make  good  decisions.  Although  advisory  groups 
have  proven  valuable  in  the  past  and  most  certainly  will  in  the  future,  their 
use  does  not  represent  a  complete  solution  to  the  problem.  It  appears  that 
the  more  profitable  approach  to  this  problem  involves  a  modification  of 
procedures  to  increase  the  objectivity  of  the  knowledgeable  and  involved 
parties  where  possible. 

Some  of  the  philosophy  used  in  the  development  and  qualification 
of  propulsion  systems  seems  appropriate  for  the  aerodynamic  portion  of 
the  evaluation  process.  It  is  understood  and  accepted  by  all  parties  that 
the  engine  will  meet  certain  specifications — some  determined  by  estab¬ 
lished  documentation,  others  added  for  the  particular  engine.  The  Air 
Force  Propulsion  Laboratory  recommends  that  the  evaluation  tests  on 
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engines  be  performed  in  government  test  facilities  (13).  The  government 
test  facility  personnel  interpret  the  data  and  make  corrections  to  account 
for  facility  or  test  pecularities.  There  are  sometimes  disagreements  be¬ 
tween  the  contractor  and  test  personnel  on  the  test  data,  but  these  are 
hopefully  resolved  with  the  assistance  of  the  SPO  on  a  technical  basis. 

Several  respondents  have  suggested  that  specifications  and  minimum 
acceptable  performance  criteria  should  also  be  established  for  the  aero¬ 
dynamic  portion  of  each  aircraft  development  program.  These  broad  speci¬ 
fications  and  areas  for  investigation  could  be  defined  for  each  decision- 
point  milestone  as  noted  by  the  responses  in  Exhibits  1  and  2,  Appendix 
D.  Additional  details  would  be  required  to  match  the  needs  and  problem 
areas  of  the  particular  aircraft.  It  is  important  that  all  anticipated  problem 
areas  involving  acceptable  data  limits,  data  correction  techniques,  etc. 
be  resolved  among  parties  early  in  the  program  before  a  specific  conflict 
arises.  Such  agreements,  removed  from  the  pressure  of  an  immediate  and 
critical  decision,  should  add  much  objectivity  to  the  results. 

There  seems  to  be  good  agreement  among  most  respondents  that  the 
government  should  determine  compliance  of  the  contractor's  product  in 
government  facilities.  There  is  also  agreement  that  these  facilities 
should  be  the  best  available  to  minimize  misinterpretation  of  the  data. 
Several  respondents  have  noted  that  test  facility  personnel  do  not  usually 
have  the  capability  of  analyzing  data  from  the  systems  standpoint;  i.  e.  , 
translation  of  the  facility  data  into  aircraft  performance  data.  However, 
it  is  expected  that  they  should  be  in  a  position  to  attest  to  the  validity  of 
the  data  taken  in  the  facility  and  to  specify  corrections  to  be  applied  to 
the  results.  The  role  of  the  DoD  Test  Center  and  test  personnel  is  clari¬ 
fied  somewhat  by  the  responses  to  Question  21,  Appendix  B.  The  respond¬ 
ent  is  asked  if  DoD  facilities  could  better  serve  the  development  programs 
by  providing  experienced  personnel  on-site  to  provide  additional  services; 
e.  g.  ,  guidance  on  the  test  program,  data  analyses,  recommendations. 
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etc.  There  were  67%  affirmative  responses  and  24%  negative  responses. 
However,  the  question  was  evidently  poorly  worded,  for  the  comments 
of  the  respondents  indicated  that  most  wanted  the  same  thing.  Most 
respondents  do  not  want  another  NASA  at  AEDC;  nor  do  they  want  test 
facility  personnel  trying  to  design  aircraft.  They  do  want  test  facility 
personnel  to  know  their  facility  better;  they  want  to  be  advised  of  the 
accuracy  of  the  data  that  comes  from  the  facility;  they  want  to  know  the 
correlation  with  flight  data;  and  they  want  to  be  advised  on  how  best  to 
establish  tests  for  the  facility. 

Some  of  the  contractor  personnel  have  indicated  a  concern  that 
they  may  become  completely  isolated  from  the  evaluation  process.  They 
fear  that  their  product  may  be  poorly  judged  through  misinterpretation  of 
data  by  the  government  testers  and  evaluators.  Since  the  contractor 
usually  has  the  most  detailed  knowledge  of  the  product  being  evaluated, 
his  expertise  and  counsel  is  essential  to  the  evaluation  process.  The 
contractor  should  have  the  opportunity  to  question  the  interpretations 
made  by  the  government  test  personnel  and  SPO.  It  may  also  be  appro¬ 
priate  to  use  outside  technical  advisory  groups  or  experts  to  supplement 
the  available  expertise  on  some  Issues  and  to  provide  another  opinion 
when  issues  are  unresolved.  The  procedures  outlined  here  are  intended 
to  apply  for  tests  in  DoD  facilities.  When  testing  is  accomplished  in 
NASA  facilities,  the  interplay  between  the  NASA  test  personnel,  SPO 
and  contractor  becomes  the  important  issue.  The  DoD  test  facility  per¬ 
sonnel  should  then  become  advisors  to  the  SPO — both  for  establishing 
the  test  plans  and  interpreting  the  results  of  the  test  facility. 

Recommendations 

1.  Almost  all  of  the  aerospace  experts  who  have  responded  to  this  study 
recognize  that  a  sound  facility  test  program  is  essential  for  successful 
fulfillment  of  the  DoD  directives  for  systems  development.  It  has  been 
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demonstrated  that  this  facet  of  policy  implementation  is  not  receiving 
sufficient  attention  and  that  there  are  strong  influences  which  work 
against  implementation. 

RECOMMENDATION:  AFSC  should  recognize  and  reduce  the  influences 
that  tend  to  force  an  inadequate  facility  test  program.  Test  program 
criteria  and  coordination  procedures  should  be  established  to  assure 
that  the  test  program  is  not  subjected  to  unacceptable  compromises  by 
other  factors  in  the  development  program. 

•  2.  It  has  been  shown  that  the  test  facility  can  provide  useful  and  tan¬ 
gible  information  to  the  government  decision-maker  for  his  Program  and 
Ratification  Decisions.  The  appropriate  utilization  of  the  test  facility 
during  both  the  Conceptual  and  Validation  Phases  has  been  enunciated 
and  illustrated  with  examples  provided  by  the  respondents. 
RECOMMENDATION:  Milestone  criteria  which  can  be  demonstrated  in 
the  test  facility  should  be  established  for  the  Program  and  Ratification 
Decisions.  All  anticipated  problem  areas  involving  acceptable  data 
limits,  data  correction  techniques,,  etc,  should  be  resolved  as  well  as 
possible  among  the  contractor,  SPO  and  test  facility  personnel  early  in 
the  program  before  a  conflict  arises.  The  contractor's  product  should 
be  evaluated  in  government  facilities  by  government  personnel.  The  test 
facilities  should  be  adequate  to  prevent  misinterpretation  of  the  data  and 
argument  over  the  results. 

3.  The  Fitzhugh  Report  (14)  recommends  that  a  separate  Defense  Test 
Agency  be  created  within  the  DoD  to  perform  the  functions  of  overview 
of  all  defense  test  and  evaluation.  It  is  not  clear  whether  test  facilities 
like  those  at  AEDC  would  be  a  part  of  the  new  Agency,  since  emphasis  in 
the  report  is  placed  more  on  operational  testing  than  functional  or  engi¬ 
neering  testing.  However,  if  the  recommendation  is  implemented  and 
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such  facilities  are  included,  the  evaluation  role  of  the  government  test 
facility  will  be  somewhat  determined.  Until  that  time,  it  appears  appro¬ 
priate  to  modify  procedures  within  AFSC  to  reduce  or  resolve  the  dilemma 
of  conflicting  interests  in  the  evaluation  process. 

RECOMMENDATION:  Attention  should  be  given  to  more  separation  of  the 
development  and  evaluation  functions  within  AFSC.  Test  personnel  should 
be  removed  when  possible  from  the  pressures  which  are  inherent  in  the 
development  process,  but  which  act  to  force  loss  of  objectivity  in  the 
evaluation  process. 

4.  Most  respondents  do  not  expect  competitive  full-scale  aircraft  flight 
demonstrations  to  play  a  significant  role  in  most  future  development  pro¬ 
grams.  This  attitude  is  caused  primarily  by  knowledge  of  the  large  expense 
of  such  dual  development  programs  for  technically  advanced  and  large 
aircraft.  Yet  it  was  generally  agreed  that  analyses  alone  provide  insuf¬ 
ficient  evidence  to  permit  selection  of  a  contractor  and  the  "best  aircraft.  " 
A  compromise  position  which  involves  competition  between  aerodynamic 
models  in  a  government  facility  was  accepted  by  many  respondents  as  a 
desirable  addition  to  criteria  for  selection  of  a  contractor. 
RECOMMENDATION:  Competitive  demonstrations  of  contractors'  aerody¬ 
namic  models  in  government  facilities  should  be  given  strong  considera¬ 
tion  when  full-scale  competition  is  not  feasible. 

5.  It  was  noted  by  several  respondents  that  the  DoD  Test  Center  (AEDC) 
personnel  do  not  provide  some  of  the  services  required  by  their  customers. 
RECOMMENDATION:  AEDC  test  facility  personnel  should  possess  (or 
develop)  the  expertise  to  advise  the  aircraft  contractor  and  SPO  on  the, 
validity  of  the  test  facility  data  and  to  specify  corrections  to  be  applied 
to  the  results.  This  is  expected  to  require  more  emphasis  on  the  corre¬ 
lation  of  facility  data  with  flight  results. 
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6.  The  Secretary  and  Deputy  Secretary  of  Defense  have  made  the  point 
that  the  SPD  should  have  the  authority  and  flexibility  to  exercise  more 
of  his  own  good  judgement  to  develop  new  weapon  systems.  It  is  ex¬ 
pected  that  the  SPD  must  be  able  to  cut  through  the  "red  tape"  and  not 
be  victimized  by  bureaucracy  (15).  Good  management  principles  dictate 
that  this  decision-making  authority  carries  with  it  the  responsibility  to 
hear  the  analyses  and  recommendations  of  the  appropriate  technical  ex¬ 
perts  in  supporting  AFSC  organizations.  The  comments  given  the  author 
lead  to  the  conclusion  that  this  has  not  always  been  the  case.  Although 
the  modern  SPO  and  ASD  are  staffed  by  some  of  the  more  qualified  mili¬ 
tary  and  civilian  Air  Force  members,  it  is  impossible  for  them  to  represent 
the  technical  expertise  available  in  the  supporting  Air  Force  organiza¬ 
tions.  Unfortunately,  the  suggestion  to  "force"  more  coordination  between 
the  SPO  and  the  supporting  organizations  has  been  interpreted  by  some  of 
the  affected  respondents  as  more  bureaucracy  and  limitation  of  the  SPD' s 
authority  and  flexibility. 

RECOMMENDATION;  Oontributions  from  the  Air  Force  organizations  which 
support  the  SPO  should  be  more  automatic;  i.  e.  ,  provided  without  request 
of  the  SPO.  More  specifically,  AFSO  Supplement  1  to  AFR  80-14  should 
be  modified  to  make  test  centers  responsible  for  certain  advice,  analyses 
and  reports  without  the  SPO  specifying  such  a  requirement.  Such  direc¬ 
tives  must  be  carefully  worded  so  that  the  supporting  organization  remains 
responsive  to  the  changing  needs  of  the  SPO. 

7.  General  George  S.  Brown,  AFSC  Commander,  has  emphasized  that  he 
wants  to  identify  "losers"  among  the  new  weapon  systems  under  develop¬ 
ment  as  soon  as  possible  and  divert  the  funds  to  "winners"  (15).  The 
evidence  obtained  in  this  study  indicates  that  a  critical  appraisal  of 
facility  test  results  and  comparison  with  specifications  is  the  best  way 
to  make  such  an  early  determination.  The  option  is  open  to  drop  the 
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system,  modify  the  specifications,  or  continue  development.  But,  the 
decision-maker  cannot  always  expect  to  have  the  necessary  information 
for  his  decision  if  the  program  is  "sold"  to  him  by  its  advocates.  He 
must  have  advantage  of  unbiased  data  to  make  his  judgement. 
RECOMMENDATION:  When  there  is  an  unresolved  difference  of  opinion 
between  the  Test  Center  and  SPO  regarding  interpretation  of  critical 
test  data,  the  Test  Center  should  have  the  opportunity  to  submit  a  report 
of  the  circumstances  to  the  Commander,  AFSC.  This  should  not  be  con¬ 
strued  as  an  attempt  to  by-pass  the  authority  of  the  JiPD  to  make  deci¬ 
sions.  It  is,  instead,  a  "safety  valve,  "  an  incentive  for  better  coordina- 
tion  and  agreement  at  lower  levels,  and  a  means  for  getting  more  objec¬ 
tivity  into  the  analyses. 


CHAPTER  V 


THE  ADEQUACY  OF  TEST  FACILITIES 

There  have  been  statements  in  previous  chapters  by  contributors 
to  this  study  who  have  either  directly  or  indirectly  questioned  the  ade¬ 
quacy  of  today's  aeronautical  test  facilities.  The  respondents'  concern 
with  the  facility's  inability  to  predict  the  "real  world"  of  flight  and  the 
uneasiness  expressed  as  a  result  of  extreme  data  adjustment  and  "cor¬ 
rection"  may  be  interpreted  as  an  indication  of  ineffective  test  results 
and  facility  inadequacy.  The  role  that  the  test  facility  is  playing  in  the 
system  development  cycle  and  the  role  that  it  is  expected  to  play  in  the 
future  demands  that  the  data  from  the  test  facility  be  accurate  and 
believable. 

Unfortunately,  facility  construction  and  modernization  is  not  keep¬ 
ing  pace  with  the  increasing  test  needs  dictated  by  more  sophisticated, 
high  performance  and  large  aircraft.  A  review  of  recent  history  will  re¬ 
veal  how  this  situation  has  arisen.  Most  of  the  large  government  test 
facilities  of  modern  vintage  were  completed  in  the  early  1950's.  It  was 
after  World  War  II  that  concern  was  expressed  in  this  country  over  the 
advanced  state  of  aeronautical  technology  in  Germany,  and  the  situa¬ 
tion  was  attributed  primarily  to  their  vastly  superior  wind  tunnels  and 
other  environmental  test  facilities.  An  investigating  committee  headed 
by  Dr.  von  Karman  produced  a  report  (16)  which  recommended  the  con¬ 
struction  of  several  facilities;  this  resulted  in  the  creation  of  the 
Arnold  Engineering  Development  Center  in  Tennessee  and  several  new 
NACA  (now  NASA)  facilities.  Fortunately,  these  facilities  were  designed 
with  great  foresight  and  imagination;  they  have  served  well  for  almost 
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two  decades.  Unfortunately,  the  availability  of  good  test  facilities  has 
become  such  an  accepted  fact  that  it  is  sometimes  difficult  to  convince 
those  not  closely  associated  with  test  facilities  that  the  facilities  are 
often  less  than  adequate. 

In  this  chapter,  the  nature  and  seriousness  of  facility  inadequacies 
and  some  possible  remedies  are  explored.  It  is  hypothesized  that  de¬ 
ficiencies  in  environmental  simulation  in  aeronautical  test  facilities 
have  had  detrimental  effects  upon  systems  development,  and  that  these 
effects  are  growing  worse  with  increasing  time.  Specific  examples  to 
illustrate  this  cause-effect  relationship  are  solicited  from  the  aerospace 
experts.  It  is  further  hypothesized  that  facility  deficiencies  are  subtly 
causing  compromises  in  aircraft  design  because  of  the  designer's  aware¬ 
ness  of  and  dependency  on  available  test  capability.  The  roles  of  the 
contractor-owned  and  government  test  facilities  are  investigated,  and 
the  adequacy  of  the  Air  Force  policy  regarding  construction  and  use  of 
contractor  facilities  is  questioned.  Finally,  the  workability  of  the  pro¬ 
cess  for  acquisition  of  major  test  facilities  within  the  Air  Force  is  con¬ 
tested. 


Consequences  of  Test  Facility  Inadequacies 

It  is  hypothesized  that  the  deficiencies  in  environmental  simula¬ 
tion  in  aeronautical  test  facilities  have  had  detrimental  effects  upon 
systems  development.  The  hypothesis  is  posed  in  Question  15,  Appendix 
B,  and  the  respondent  is  asked  to  indicate  his  agreement.  The  results 
are  shown  in  Table  13. 

Nine  per  cent  of  the  respondents  believe  that  facility  deficiencies 
have  so  far  had  no  detrimental  effects  upon  aerospace  systems  develop¬ 
ment  programs.  Some  of  the  "none"  answers  were  qualified  with  comments 
such  as  the  following:  "The  answer  is  based  on  what  facilities  presently 
exist  and  the  marginal  cost  associated  with  simulation  improvement. 
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TABLE  13 

REPLIES  TO  HYPOTHESIS  THAT  DEFICIENCIES  IN 
ENVIRONMENTAL  SIMULATION  IN  AERONAUTICAL 
TEST  FACILITIES  HAVE  HAD  DETRIMENTAL 
EFFECTS  ON  SYSTEMS  DEVELOPMENT 


Agency 

None 

Some 

Much 

Very 

Much 

No  Answer 

Industry 

6 

40 

8 

0 

4 

ASD 

3 

10 

2 

0 

3 

Headquarters 

1 

5 

1 

0 

0 

AFSC  Labs 

1 

6 

2 

0 

1 

AEDC 

0 

8 

3 

2 

0 

NASA 

0 

5 

3 

0 

0 

Other 

0 

1 

2 

0 

0 

Totals 

11 

75 

21 

2 

8 

Money  to  significantly  improve  environmental  simulation  could  better  be 
spent  elsewhere."  The  author  had  not  expected  the  respondents  to  argue 
the  question,  since  the  difference  between  the  desired  and  actual  magni¬ 
tude  of  certain  simulation  parameters  is  a  matter  of  record  and  well 
publicized.  However,  two  of  the  respondents  who  answered  "none"  did 
not  acknowledge  the  problem.  One  comment  was:  "We  have  more  test 
capability  (quantity  and  quality)  than  we  can  possibly  use  for  the  next 
ten  years.  There  are  a  few  minor  exceptions.  "  However,  64%  have 
experienced  some  effect  and  about  20%  believe  there  is  much  or  very 
much  detrimental  effect  on  systems  development  as  a  result  of  facility 
deficiencies.  Since  the  respondents  are  primarily  composed  of  individuals 
who  have  been  instrumental  in  either  developing  or  testing  most  of  the 
aircraft  developed  in  this  country  during  the  last  fifteen  years,  it  would 
appear  that  the  testing  deficiencies  are  real  and  are  causing  undesirable 
effects.  The  extent  and  seriousness  of  these  effects  are  impossible  to 
determine  from  Table  13,  so  the  respondents  were  requested  to  comment 
on  specific  deficiencies  and  give  examples  where  aircraft  development 
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programs  have  been  adversely  affected  because  of  inadequate  test  facil¬ 
ities  and  poor  environmental  simulation.  Almost  all  those  who  made 
comments  chose  to  relate  situations  which  represent  present  problems; 
i.  e. ,  the  facility  deficiencies  exist  today  and  are  getting  worse. 

The  deficiency  most  referenced  was  the  lack  of  high  Reynolds 
number  capability  at  transonic  speeds.  The  respondents  noted  serious 
deficiencies  in  the  ability  of  the  wind  tunnel  to  produce  results  which 
would  permit  prediction  of: 

Full-scale  shock/boundary  layer  interaction  effects, 

High-lift  performance. 

Transonic  buffet  boundary. 

Tail  Loads. 

Most  respondents  agree  that  the  inability  to  get  Reynolds  number  near 
full  scale  for  large  subsonic  aircraft  is  a  problem.  The  specific  aircraft 
program  most  often  mentioned  was  the  Lockheed  C-141,  where  the  facil¬ 
ity  inadequacy  resulted  in  difficulties  in  predicting  the  chord-wise 
pressure  distribution  on  the  wing  at  high  subsonic  speeds,  thus  providing 
erroneous  estimates  for  pitching  moment  and  structural  design. 

Another  frequently  mentioned  effect  of  a  facility  deficiency  was  the 
inability  to  investigate  the  areas  of  inlet-engine  compatibility  and  noz¬ 
zle-airframe  integration.  The  F-111  had  development  problems  in  these 
areas  which  were  difficult  to  cure  because  of  scaling  problems  (model 
size,  Reynolds  number,  sting  corrections,  and  propulsion  airflow  simu¬ 
lation).  This  made  it  quite  difficult  to  simulate  the  actual  boundary 
layer,  shock  Interactions  and  external  pressures  which  are  present  on 
the  full  scale  aircraft. 

Although  this  study  does  not  directly  address  the  development  of 
the  propulsion  system,  the  third  most  frequently  noted  deficiency  was 
the  inability  of  the  test  facilities  to  support  development  of  larger  and 
more  powerful  subsonic  and  supersonic  turbofan  and  turbojet  engines. 
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It  was  noted  that  undesirable  compromises  have  already  been  made  in 
engine  evaluation  in  facilities,  and  that  the  situation  is  certain  to  get 
worse  as  engines  grow  in  thrust  and  size. 

Another  often  mentioned  inadequacy  in  facilities  was  the  almost 
complete  lack  of  good  development  facilities  for  hypersonic  aircraft 
(i.  e. ,  above  a  Mach  number  of  5).  It  was  noted  that  uncertainties  in 
heat  transfer  data  are  impacting  on  the  Space  Shuttle  development  pro¬ 
gram.  The  inability  to  determine  aerodynamic/thermodynamic  interaction 
effects  at  these  speeds  concerns  many  respondents.  It  was  the  opinion 
of  several  that  the  development  of  hypersonic  engines  has  been  com¬ 
pletely  stymied  by  lack  of  adequate  facilities.  Again,  Reynolds  number 
simulation  is  inadequate  in  this  speed  regime  as  well.  It  was  also  noted 
that  the  development  of  higher  performance  re-entry  vehicles  for  ICBM's 
has  been  essentially  abandoned  because  facilities  of  adequate  perform¬ 
ance  are  not  available.  Dependence  on  flight  testing  has  proven  pro¬ 
hibitively  expensive  and  has  not  provided  the  needed  data. 

A  few  of  the  other  more  frequently  mentioned  facility  needs  have 
been  abbreviated  and  listed  below: 

--  Facilities  which  do  not  experience  transonic  blockage  (at  high 
angle  of  attack)  for  testing  maneuvering  aircraft. 

--  Facilities  which  provide  a  low  turbulence  level  for  airfoil  research. 

--  Facilities  which  provide  Mach/Reynolds  number  simulation  con¬ 
currently  for  re-entry  vehicles. 

--  Facilities  to  develop  VTOL/STOL  aircraft — particularly  for  "power- 
on"  testing. 

—  Facilities  to  test  spin  characteristics  of  modern  aircraft. 

Many  respondents  have  also  provided  some  technical  detail  in  sup¬ 
port  of  their  comments.  Such  detail  is  omitted  for  the  purposes  of 
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this  report. 

The  preceding  discussion  should  convince  the  reader  that  facility 
deficiencies  are  real  and  are  causing  problems  in  systems  development. 
However,  many  respondents  have  also  expressed  concern  over  the  in¬ 
adequacy  of  testing  techniques  within  existing  facilities.  Most  users 
of  the  facility  would  like  to  see  large  improvements  in  ability  to  correct 
data  for  effects  caused  by  the  model  mounting  system,  jet  engine  flow/ 
wake,  slotted  and  porous  walls,  etc.  There  is  a  strong  demand  for  more 
emphasis  on  correlation  of  wind  tunnel  data  with  flight  data.  The  respond¬ 
ents  appear  to  be  saying  that  there  are  many  desirable  and  necessary 
improvements  which  can  be  made  to  enhance  the  quality  of  facility  test 
data  which  do  not  necessitate  new  facility  construction.  It  is  unfor¬ 
tunate  that  more  emphasis  is  not  placed  on  such  improvements,  for  the 
pay-off  per  dollar  spent  would  be  significant.  However,  this  type  of 
research  has  never  been  particularly  popular,  and  recent  trends  toward 
reduced  funding  for  testing  facilities  will  probably  further  reduce  such 
"overhead"  expenditures. 

Several  respondents  have  noted  that  the  effect  of  test  facility  de¬ 
ficiencies  on  past  system  development  is  inconsequential  compared  to 
what  we  may  expect  in  the  future.  They  say  that  such  inadequacies  are 
already  causing  technology  to  be  stifled  so  that  the  knowledge  available 
to  the  aircraft  designer  is  less  than  required.  Question  16,  Appendix  B, 
addresses  this  dilemma  and  asks  the  respondent  what  effect  he  expects 
on  aeronautical  systems  development  over  the  next  decade  if  test  facility 
capability  is  not  improved.  As  one  would  expect  from  the  discussion  thus 
far,  most  respondents  anticipate  more  costly  and  higher  risk  development 
programs  and  less  than  desired  performance  in  future  aircraft  as  a  result 
of  the  testing  deficiencies.  Of  course,  specifics  are  dependent  upon  the 
directions  that  future  aeronautical  developments  take  in  this  country.  As 
in  the  responses  to  the  preceding  question,  a  very  few  respondents  have 


71 


not  observed  any  facility  deficiencies  in  the  past  and  do  not  expect  any 
in  the  future.  Several  replies  reflect  the  "marginal"  considerations  of 
the  expert.  For  example,  the  respondent  may  be  in  favor  of  additional 
facilities  in  "new"  technical  areas  to  provide  options  to  the  designer. 
However,  he  is  against  improving  existing  facilities  with  additional 
capability;  to  him  it  does  not  appear  cost-effective  or  necessary  on 
technical  grounds.  One  respondent  made  the  interesting  point  that  the 
effects  of  higher  risks  and  more  costly  programs  brought  on  by  facility 
deficiencies  would  further  add  to  the  disenchantment  between  the  public/ 
Congress  and  the  military/aerospace  industry.  Some  of  the  respondents' 
comments  which  represent  the  majority  opinions  are  quoted  below  to 
illustrate  the  expected  results  of  facility  inadequacies  in  the  next  decade. 

"The  full  benefits  of  potential  aerodynamic  breakthrough  (e.  g.  ,  super¬ 
critical  airfoils)  will  not  be  realized  because  of  poor  simulation  of 
flight  conditions  such  that  less  than  optimum  performance  will  be  ob¬ 
tained.  Development  programs  will  be  more  costly  than  necessary 
because  of  poor  correlation  between  tunnel  and  flight  testing — result¬ 
ing  in  added  tests  to  correct  design  deficiencies.  " 

"Systems  will  likely  require  more  extensive  changes  during  flight  develop¬ 
ment  to  achieve  predicted  performance.  " 

"Unforeseen  problems  will  be  discovered  too  late  for  economical  solu¬ 
tion.  " 

"Definitely  more  costly  development,  which  will  in  turn  limit  the  per¬ 
formance  of  aircraft,  either  through  failure  to  meet  specifications,  or 
through  lowered  performance  specifications  designed  to  keep  develop¬ 
ment  costs  from  becoming  excessive." 

"Longer,  more  costly  development  programs  producing  less  than  optimum 
designs.  Fewer  programs  because  of  budget  constraints.  " 

"Designers  will  play  it  conservative  with  little  advanced  technology. 

There  will  be  some  real  bloopers  that  show  up  in  flight-$  $  $  $.  " 

"Structurally,  the  designs  will  be  conservative--overweight  and  more 
costly.  " 
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"We  will  end  up  collecting  the  data  in  flight  tests;  i.  e. ,  the  most  expen¬ 
sive  and  least  precise  way. " 

A  further  hypothesis  concerning  the  consequencles  of  test  facility 
inadequacies  is  that  this  lack  of  test  capability  is  subtly  causing  com¬ 
promises  in  aircraft  design  because  the  designer  is  aware  of  the  limits 
of  facility  capability  and  sometimes  accepts  a  compromise  design  because 
he  cannot  verify  new  ideas  with  any  degree  of  confidence.  The  hypothesis 
is  addressed  in  Question  22,  Appendix  B,  and  the  respondents'  agreement 
is  noted  in  Table  14.  Fifty-six  per  cent  of  the  respondents  reject  the  hy¬ 
pothesis,  15%  very  strongly.  Only  23%  accept  the  hypothesis.  Many  of 
those  who  have  disagreed  listed  numerous  examples  where  large  technical 
risks  have  been  taken  in  previous  aerospace  designs  without  adequate 
facility  support.  However,  many  of  those  who  agree  with  the  proposition 
acknowledge  the  hypothesized  tendency  and  some  respondents  admit  know¬ 
ledge  of  such  compromises.  A  review  of  the  comments  just  quoted  in 
response  to  Question  16  will  show  that  several  of  these  comments  indi¬ 
cate  a  concern  over  conservative  designs  and  reduced  advanced  technol¬ 
ogy  expected  in  future  aircraft  as  a  result  of  test  facility  deficiencies. 

The  author  concludes  that  the  facility  deficiencies  are  causing  design 
compromise,  though  not  yet  very  extensive.  Such  a  trend  has  serious 
overtones,  for  it  quietly  leads  to  mediocrity.  The  problem  is  difficult  to 
identify  and  correct,  because  it  does  not  appear  as  a  direct  conflict  be¬ 
tween  test  requirements  and  facility  capability. 

The  author  has  observed  that  similar  tests  are  sometimes  performed 
in  different  test  facilities  in  an  attempt  to  verify  test  data  which  is  in 
doubt  because  of  the  uncertainty  brought  about  by  the  extrapolation  of 
simulation  parameters.  The  respondent  is  asked  in  Question  26,  Appen¬ 
dix  B,  to  use  his  experience  to  make  an  estimate  of  the  share  of  the 
modern  test  program  that  is  motivated  by  such  facility  deficiencies.  Note 
that  the  question  excludes  those  tests  performed  in  different  facilities 
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to  acquire  different  types  of  data.  Only  44%  of  the  respondents  who  com¬ 
pleted  the  questionnaire  of  Appendix  B  answered  this  question  quantita¬ 
tively.  The  answers  varied  from  1%  to  100%;  however,  all  but  three  of 
the  responses  were  between  5%  and  30%.  The  author  felt  that  the  ex¬ 
treme  three  answers  were  a  result  of  misinterpretation  of  the  question, 
so  they  were  dropped  for  treatment  of  the  data.  The  data  have  a  mode 
of  10%,  a  median  of  about  20%,  and  a  mean  of  about  17%. 


TABLE  14 

AGREEMENT  WITH  HYPOTHESIS  THAT  FACILITY  DEFICIENCIES 
ARE  SUBTLY  CAUSING  COMPROMISES  IN  AIRCRAFT 
DESIGN  BECAUSE  OF  THE  DESIGNER'S  AWARENESS 
OF  AVAILABLE  FACILITY  CAPABILITY 


Agency 

Strongly 

Disagree 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

8 

24 

9 

16 

1 

ASD 

6 

8 

3 

0 

1 

Headquarters 

0 

3 

3 

1 

0 

AFSC  Labs 

2 

7 

0 

1 

0 

AEDC 

1 

2 

6 

2 

1 

NASA 

0 

3 

2 

3 

0 

Others 

0 

2 

1 

0 

0 

Totals 

17 

49 

24 

24 

3 

The  question  of  the  previous  paragraph  was  asked  with  regard  to 
specific  aircraft  development  programs  in  Question  18,  Appendix  C.  The 
intent  was  to  find  the  change  in  the  answers  with  time,  with  the  hypothe¬ 
sis  being  that  the  effects  of  facility  inadequacies  should  be  getting  worse 
with  increasing  time.  The  data  gave  more  of  a  "U-shaped"  curve  than  a 
trend  line.  It  is  true  that  more  of  the  modern  aircraft  development  pro¬ 
grams  have  experienced  duplicate  testing;  however,  in  the  early  1950 's 
the  lack  of  good  transonic  test  facilities  caused  the  same  results.  Of  the 
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aircraft  reviewed,  60%  of  the  evaluators  indicated  duplicate  testing  of 
the  aircraft  of  the  1960's;  only  about  30%  of  the  evaluators  indicated 
such  testing  for  the  pre-1960  aircraft.  The  magnitude  of  the  estimates 
were  close  to  those  given  in  the  last  paragraph,  with  a  mean  of  about 
15%.  It  is  tempting  to  conclude  that  the  significant  increase  in  dupli¬ 
cate  testing  in  the  1960's  proves  the  hypothesis.  However,  there  are 
other  factors  which  may  influence  the  data.  The  types  of  aircraft  in¬ 
cluded  in  each  time  period  is  important,  for  low-performance  or  conven¬ 
tional  aircraft  designs  do  not  usually  have  this  problem.  The  avail¬ 
ability  of  more  facilities  and  the  opportunity  to  exercise  such  check- 
tests  in  the  later  time  period  may  prejudice  the  data;  or  perhaps  the 
data  simply  reflect  a  change  in  test  philosophy.  The  author  will  only 
conclude  from  the  responses  that  about  15%  of  the  facility  testing  in 
modern  aircraft  development  programs  appears  to  be  caused  by  the  in¬ 
ability  of  test  facilities  to  provide  the  desired  simulation  parameters. 
The  primary  cause  given  by  most  respondents  for  duplicate  testing  was 
inadequate  Reynolds  number. 

Many  of  the  responses  to  the  questions  concerning  facility  inade¬ 
quacies  were  qualified  or  appeared  to  be  somewhat  guarded.  Although 
most  of  the  respondents  acknowledged  the  facility  deficiencies  and  the 
problems  they  are  causing,  they  seemed  concerned  that  this  problem 
would  be  taken  out  of  perspective  with  some  of  the  other  facility- 
related  problems.  The  respondents'  advice  (as  interpreted  by  the 
author)  is  to  attack  the  problems  associated  with  test  facilities  in  the 
following  order  of  priority: 

1.  Make  better  use  of  existing  facilities  through  better  and  more 
timely  test  programs. 

2.  Perform  the  necessary  studies  and  experiments  to  permit  better 
extrapolation  of  data  from  existing  facilities  to  the  flight 
environment. 
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3.  Build  those  few  well-identified  facilities  where  there  is  gen¬ 
eral  agreement  on  the  deficiency  and  need  for  the  data. 

Contractor  and  Government  Owned  Facilities 

Most  of  the  airframe  industrial  contractors  have  their  own  aerody¬ 
namic  test  facilities.  Although  some  contractor  test  facilities  are  more 
extensive  than  others,  they  are  all  usually  "small  and  inexpensive"  when 
compared  with  the  large  development  test  facilities  available  within  the 
government.  The  theory  has  been  that  a  contractor  needs  readily  avail¬ 
able  facilities  in  his  own  backyard  for  a  large  quantity  of  preliminary 
cut-and-try  testing.  It  is  expected  that  further  downstream  in  the  de¬ 
velopment  program  additional  refinement  and  evaluation  of  design  will 
require  more  sophisticated  test  facilities  which  are  beyond  the  financial 
capacity  of  a  single  contractor.  The  government  (primarily  AEDC  and 
NASA)  has  assumed  the  role  of  providing  this  more  expensive  "national" 
test  capability;  however,  there  are  alternatives  to  this  situation.  The 
now  defunct  Southern  California  Cooperative  Wind  Tunnel  was  jointly 
owned  and  used  by  several  companies  and  operated  by  an  independent 
organization  (California  Institute  of  Technology).  It  was  a  rather  large 
and  extremely  productive  wind  tunnel  until  the  "missile  era"  caused  its 
demise.  A  sister  facility  exists  at  the  Cornell  Research  Laboratory  and 
is  operated  for  any  user  on  a  profit-making  basis. 

The  respondents  have  been  questioned  to  determine  their  position 
regarding  the'types"  of  facilities  industry  and  government  should  possess, 
and  to  establish  some  consensus  on  the  roles  of  the  industry  and  govern¬ 
ment  facilities  in  aerospace  development  programs.  Question  17,  Appen¬ 
dix  B,  seeks  to  determine  agreement  with  the  situation  where  industry 
has  the  high-use/low-cost  facilities  and  government  has  the  high-cost/ 
low-use  facilities.  The  respondents'  answers  are  noted  in  Table  15. 

Some  of  the  respondents  correctly  point  out  that  the  government  facility 
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does  not  necessarily  have  to  be  low-use  because  it  is  high-cost.  The 
author  intended  for  the  "use"  terms  to  apply  to  an  individual  develop¬ 
ment  project,  not  to  refer  to  overall  utilization  of  the  facility.  One 
would  expect  the  few  sophisticated  facilities  to  be  used  by  a  number  of 
different  projects  feeding  in  from  the  many  "low-cost"  facilities.  The 
seemingly  substantial  endorsement  of  this  proposition  (80%  agreement) 
must  be  interpreted  with  care,  for  there  are  many  qualifications  among 
the  agreements.  Some  of  the  comments  given  in  response  to  Question 
17  have  been  analyzed  and  are  explained  in  the  following  discussion. 


TABLE  15 

RESPONDENTS'  AGREEMENT  WITH  PROPOSITION  THAT 
INDUSTRY  POSSESS  HIGH-USE/LOW-COST  TEST 
FACILITIES  AND  GOVERNMENT  PROVIDE 
LOW- USE/HIGH-COST  TEST  FACILITIES 


Agency 

Disagree 

Strongly 

Disagree 

No 

Opinion 

Agree 

Agree 

Strongly 

Industry 

2 

2 

6 

32 

16 

ASD 

2 

2 

1 

10 

3 

Headquarters 

1 

1 

1 

1 

3 

AFSC  Labs 

0 

0 

0 

8 

2 

AEDC 

0 

4 

0 

8 

1 

NASA 

0 

0 

0 

3 

5 

Other 

,0 

1 

0 

2 

0 

Totals 

5 

10 

8 

64 

30 

Almost  all  industry  respondents  believe  that  industry  should  have 
its  own  low-cost  backyard  facilities.  They  argue  that  such  facilities 
provide  quick  response  and  flexibility  to  the  contractor,  and  are  the  key 
to  industry  research  and  contractor  funded  early  development  prior  to 
government  involvement.  Industry  claims  that  without  their  own  facilities 
to  support  innovative  development  and  provide  immediate  access  for 
investigation  of  design  problems,  the  development  programs  would 
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increase  In  cost  because  of  unavailability  of  government  wind  tunnels, 
higher  cost  of  obtaining  data,  remote  site  inconvenience,  etc.  Most  of 
the  government  respondents  also  agree  that  industry  should  have  these 
low-cost  backyard  facilities,  but  sometimes  with  resignation  rather 
than  the  enthusiasm  shown  by  industry.  Most  government  respondents 
believe  that  the'fcontractor  owned"  facilities  are  essentially  paid  for 
by  the  government  as  the  charges  and  over-head  are  prorated  over  a 
number  of  aircraft  development  projects.  There  is  a  strong  feeling  with¬ 
in  the  government  that  there  are  presently  too  many  contractor  facilities 
and  not  sufficient  work  to  keep  the  facilities  busy.  It  is  argued  that 
contractor  facilities  are  not  as  readily  available  to  all  potential  users 
as  those  operated  by  the  government  (primarily,  because  of  proprietary 
considerations).  There  is  substantial  agreement  among  government 
respondents  that  the  government  needs  better  control  on  the  number  of 
such  facilities  at  industrial  sites  developed  indirectly  with  DoD  funds. 
Yet,  most  government  respondents  concede  that  these  facilities  are  part 
of  the  present  system  of  development,  represent  a  part  of  our  national 
capability  to  develop  aircraft,  and  therefore  should  be  used.  Some  gov¬ 
ernment  respondents  (particularly  those  associated  with  research)  point 
out  that  the  government  also  needs  some  low-cost  research  facilities  to 
support  in-house  research. 

The  industry  agrees  that  no  single  contractor  should  have  to  pro¬ 
vide  the  type  of  high-cost  facility  usually  supplied  by  the  government. 
However,  a  few  industrial  respondents  would  rather  see  several  aircraft 
companies  fund  and  support  a  large  cooperative  wind  tunnel  than  have 
the  same  constructed  and  operated  by  the  government.  They  argue  that 
they  are  not  in  favor  of  the  government  taking  over  a  role  which  they 
believe  to  be  properly  that  of  the  contractor.  They  claim  better  response 
and  cheaper  data  from  such  cooperative  tunnels.  However,  most  indus- 
rlal  respondents  accept  the  government  facilities  as  fulfilling  a  useful 
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role  and  have  come  to  depend  upon  them. 

The  government  respondents  indicated  good  agreement  on  the 
handling  of  large  test  facilities;  i.  e. ,  they  believe  in  government 
ownership  and  operation  and  explain  that  the  large  test  facilities  are 
often  one-of-a-kind  and  everyone  should  have  access  to  them.  Most 
government  respondents  believe  that  the  government  must  have  these 
sophisticated  tools  to  evaluate  and  validate  the  contractor's  product; 
i.  e.  ,  the  government  must  insist  on  qualification  of  critical  components 
in  its  own  facilities.  Many  feel  that  the  government  would  indirectly 
pay  for  the  cooperative  facility  even  if  the  contractors  pooled  their  own 
funds  to  build  it.  In  fact,  some  argue  that  it  is  not  a  question  of  who 
pays  for  the  industry  test  facilities,  but  only  a  matter  of  how  they  are 
funded. 

It  is  hypothesized  that  the  contractor-owned  aeronautical  test 
facilities  assist  the  company  in  obtaining  government  contracts.  The 
respondent  is  asked  to  review  his  experiences  and  express  his  opinion 
on  the  degree  of  influence  the  contractor's  test  facilities  have  on  con¬ 
tractor  selection  in  the  first  part  of  Question  19,  Appendix  B.  The 
results  are  shown  in  Table  16.  All  agencies  agree  that  the  contractors' 
test  facilities  are  an  important  part  of  his  competitive  "strength"  as  he 
vies  for  aerospace  development  contracts.  With  such  strong  conviction 
on  the  part  of  both  the  industrial  bidder  and  government  evaluator  as  to 
the  importance  of  the  contractor's  test  facilities,  there  can  be  little 
doubt  that  industry  is  provided  with  an  incentive  to  continue  to  build 
test  facilities  and  even  compete  to  provide  the  "best"  facility. 

The  second  part  of  Question  19  probes  a  somewhat  more  discrim¬ 
inative  position;  i.  e.  ,  does  the  response  in  Table  16  hold  true  even 
though  the  same  test  capability  is  available  for  the  company  to  use  at 
a  government  test  facility.  See  Table  17  for  the  answers.  It  would 
appear  that  the  availability  of  government  facilities  to  perform  develop- 
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ment  testing  has  little  or  no  influence  on  a  contractor's  incentive  to 
provide  his  own  facilities  and  even  duplicate  government  test  capability. 
It  may  be  concluded  that  a  very  competitive  situation  has  been  created 
which  tends  to  expand  the  contractor  facilities. 


TABLE  16 

RESPONDENTS'  AGREEMENT  THAT  CONTRACTOR-OWNED 
TEST  FACILITIES  ASSIST  THE  COMPANY  IN 
OBTAINING  GOVERNMENT  CONTRACTS 


Agency  None  Some 

Much 

Very  Much 

No  Answer 

Industry  4 

19 

17 

17 

1 

ASD  1 

9 

2 

6 

0 

Headquarters  0 

4 

2 

1 

0 

AFSC  Labs  0 

2 

5 

3 

0 

AEDC  0 

5 

4 

4 

0 

NASA  0 

2 

2 

4 

0 

Other  0 

1 

2 

0 

0 

Totals  5 

42 

34 

35 

1 

TABLE  17 

RESPONDENTS'  AGREEMENT  WITH  HYPOTHESIS  THAT  COMPANY-OWNED 

TEST  FACILITIES  HELP  THE  COMPANY  IN  OBTAINING 

GOVERNMENT  CONTRACTS  EVEN  THOUGH  THE 

SAME  TEST  CAPABILITY  IS  AVAILABLE  FOR 

COMPANY  USE  AT  A  GOVERNMENT  TEST  FACILITY 

Agency 

Yes 

No 

No  Answer 

Industry 

49 

2 

7 

ASD 

15 

1 

2 

Headquarters 

6 

1 

0 

AFSC  Labs 

10 

0 

0 

AEDC 

13 

0 

0 

NASA 

7 

1 

0 

Other 

3 

0 

_0_ 

Totals 

103 

5 

9 
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As  long  as  the  aerospace  business  has  been  prosperous,  both  the 
government  and  industrial  facilities  have  had  sufficient  workloads  to 
prevent  a  conflict  from  arising.  Now  that  there  is  a  depression  and  the 
testing  workloads  are  substantially  reduced,  it  has  become  evident  that 
the  quantity  of  facilities  (though  not  quality)  surpasses  the  need.  Gov¬ 
ernment  managers  have  begun  to  seek  controls  to  curtail  construction  of 
facilities  at  industrial  sites  and  to  direct  testing  to  government  facili¬ 
ties.  Operation  under  service  funding  within  AFSC  (particularly  AEDC) 
has  further  encouraged  efforts  to  direct  development  testing  on  AFSC 
programs  to  Air  Force  test  facilities. 

For  quite  some  time  the  government  has  had  a  policy  which  would 
limit  construction  of  new  facilities  at  a  contractor's  plant  and  direct 
testing  to  AFSC  facilities.  The  following  are  quotes  from  AFSC  Supple¬ 
ment  1  to  AFR  80-14; 

"Priority  will  be  given  to  accomplishing  tests  at  AFSC  installations.  " 

"Use  of  other  agency  or  contractor  facilities  will  be  limited  to  those 
efforts  which  cannot  be  accomplished  in  AFSC  facilities  or  which, 
in  the  best  interests  of  the  government  and  the  program  should  be 
accomplished  elsewhere.  " 

"To  promote  multiprogram  use  and  the  development  of  AFSC  cap¬ 
abilities,  new  facilities  will  be  located  on  AFSC  installations 
unless  there  are  clear  and  compelling  reasons  for  their  location 
elsewhere.  "  (6) 

The  reader  will  observe  sufficient  freedom  in  each  of  the  three  quotes 
above  to  permit  a  wide  latitude  in  implementation  of  the  policy.  The 
test  facility  construction  referenced  here  is  that  which  would  support 
and  be  funded  by  a  particular  aircraft  development  program. 

Although  deep  concern  has  been  expressed  by  many  government 
respondents  regarding  the  continued  proliferation  of  test  facilities  by 
industry,  the  government  has  no  direct  control  over  Industry  in  this 
regard.  If  an  aircraft  company  feels  that  it  needs  a  test  facility,  de- 
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termines  that  it  will  be  placed  in  a  better  competitive  position  with  the 
facility,  and  is  willing  to  risk  its  own  funds  to  build  it,  the  government 
has  no  authority  to  say  otherwise.  The  only  control  which  the  govern¬ 
ment  might  exercise  is  the  "weight"  given  for  the  facility  in  evaluation 
of  a  bid  proposal,  acceptance  or  non-acceptance  of  overhead  for  the 
facility  in  a  contract,  and  permission  to  use  the  facility  in  a  military 
aircraft  development  program. 

There  appears  to  be  no  set  of  solutions  to  the  problems  created  by 
declining  test  workloads  and  the  resulting  excess  in  availability  of  cer¬ 
tain  types  of  aerospace  facilities.  The  smaller  the  development  workload 
gets,  the  more  difficult  even  a  compromise  position  becomes.  However, 
the  respondents  have  freely  shared  with  the  author  their  opinions  and 
feelings  concerning  these  forces  and  events.  Perhaps  an  explanation 
of  some  of  these  views  will  permit  the  reader  to  better  understand  the 
situation  and  accept  the  recommendations  to  be  presented  later  in  this 
chapter. 

The  contractor  argues  that  he  is  sometimes  forced  into  AFSC 
facilities  (AEDC)  that  are  too  big,  sophisticated  and  expensive  for  his 
test  needs.  He  claims  savings  in  cost  and  time  by  being  able  to  go  to 
test  facilities  of  his  choosing  which  provide  only  the  information  required. 
Further,  he  would  like  to  use  his  own  test  facilities  as  much  as  pos¬ 
sible;  they  are  convenient  and  this  is  the  way  he  pays  for  them.  The 
SPD  tends  to  agree  with  these  arguments  by  the  contractor.  He  is 
usually  laboring  under  limited  funds  to  develop  an  aircraft;  he  certainly 
does  not  want  to  spend  extra  dollars  for  test  data  if  it  can  be  obtained 
at  less  cost.  The  fact  that  NASA  can  provide  testing  at  no  charge  to  the 
SPD  (see  Chapter  III)  further  complicates  the  situation.  Many  industrial 
and  ASD  respondents  deeply  resent  the  fact  that  they  are  being  forced  to 
AFSC  facilities,  as  they  say,  "to  keep  them  open.  " 

Now  for  another  point  of  view;  one  which  does  not  really  contradict 
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the  views  expressed  above,  but  one  which  leads  to  different  conclusions. 
The  AFSC  owned  and  operated  test  facilities  represent  a  very  substan¬ 
tial  investment;  overall,  they  are  the  best  and  most  capable  facilities 
available  for  aircraft  development.  Proficient  test  crews  have  been 
assembled  and  represent  a  valuable  resource  to  the  nation.  The  govern¬ 
ment  facilities  are  required  for  verification  and  qualification  of  the  con¬ 
tractor's  product;  attempts  to  ignore  this  step  have  sometimes  led  to 
unsatisfactory  and  expensive  consequences.  The  unfortunate  situation 
is  that  there  just  is  not  enough  test  work  in  today's  aerospace  environ¬ 
ment  to  keep  all  the  facilities  busy;  all  are  going  to  suffer,  some  more 
than  others.  The  government  facilities  must  maintain  sufficient  work¬ 
load  to  keep  together  an  experienced  technical  workforce;  if  this  is 
diluted,  their  effectiveness  and  an  important  national  capability  is  lost. 
Even  though  the  SPD  may  feel  that  he  is  sometimes  spending  more  money 
for  testing,  from  a  larger  viewpoint  the  Air  Force  may  actually  be  spend¬ 
ing  less.  Since  the  government  facilities  must  be  maintained  for  the 
reasons  stated,  it  costs  very  little  more  to  use  them  fully  than  only 
partially.  It  is  only  the  accounting  system  that  makes  their  use  appear 
uneconomical. 

A  review  of  the  results  of  Question  18,  Appendix  B,  will  conclude 
the  discussion  of  government  and  contractor  owned  facilities.  The 
question  assumes  agreement  of  some  division  between  contractor  and 
government  facilities.  It  asks  the  respondent  for  his  estimate  of  this 
dividing  line  in  terms  of  facility  construction  costs,  or  requests  sug¬ 
gestions  for  some  other  designation  of  a  division.  A  few  respondents 
from  both  industry  and  government  indicated  their  belief  that  industry 
should  build  what  it  feels  it  can  afford.  They  argue  that  the  facility 
should  depend  upon  what  program  it  is  expected  to  support,  what  future 
use  is  expected,  etc.  A  few  respondents  suggested  a  dividing  line 
based  on  charge  rate;  i.  e. ,  cost  per  hour  of  operation.  Over  one-half 
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of  the  respondents  claimed  no  opinion,  and  49  respondents  provided 
an  estimate  based  on  construction  cost  as  requested.  The  results 
ranged  from  one-half  million  dollars  to  25  million  dollars.  The  data 
have  a  mean  of  7.4  million  dollars,  a  median  of  just  under  5  million 
dollars,  and  a  mode  of  5  million  dollars. 

The  Test  Facility  Acquisition  Process 

It  does  not  appear  that  the  government  is  fulfilling  its  role  in  pro¬ 
viding  the  new,  more  sophisticated  aeronautical  test  facilities  as 
expected  by  the  aerospace  community.  Yet,  it  was  demonstrated  earlier 
in  this  chapter  that  the  requirement  for  certain  test  capability  is  well 
recognized  within  both  government  and  industry,  and  that  deficiencies 
in  the  test  facilities  have  caused  and  are  expected  to  cause  undesirable 
effects  in  aircraft  development  programs.  It  is  the  purpose  of  the  fol¬ 
lowing  discussion  to  explore  the  circumstances  that  have  produced  this 
state  of  affairs. 

It  is  during  the  Conceptual  Phase  of  development  (see  Figure  1) 
that  system  planners  first  direct  their  attention  to  the  test  facilities  that 
will  be  required  for  support  of  a  particular  system's  development.  It  is 
after  an  affirmative  Program  Decision  that  serious  attempts  are  begun  to 
provide  these  facilities.  As  shown  earlier,  much  facility  testing  is 
required  in  the  Conceptual  Phase  and  a  very  important  role  is  played  by 
the  facility  in  several  capacities  in  the  Validation  Phase.  Several  air¬ 
craft  designers  have  declared  that  their  design  progress  is  usually  paced 
by  data  from  the  facility.  Clearly,  no  test  facility  construction  or  im¬ 
provement  sponsored  by  a  new  system  can  provide  help  for  that  system 
in  the  Conceptual  Phase  testing.  Because  of  the  time  it  takes  to  design, 
construct  and  check-out  a  facility,  the  same  can  be  said  for  the  Valida¬ 
tion  Phase  (except  for  rapid  and  usually  minor  modifications).  It  is 
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obvious  that  the  facility  development  cannot  parallel  the  aircraft  develop¬ 
ment;  a  major  new  test  facility  may  require  five  to  seven  years  to  design, 
construct  and  bring  to  operational  usefulness. 

The  only  channel  available  for  acquisition  of  major  test  facilities 
within  the  Air  Force  outside  the  systems  acquisition  cycle  is  the  Mili¬ 
tary  Construction  Program  (MCP).  This  program  provides  funds  for  all 
types  of  construction  at  military  installations  (buildings,  equipment, 
appurtenances,  utilities,  etc.).  Submittal  through  this  channel  takes 
the  technical  facility  out  of  direct  association  with  the  systems  it  would 
support  and,  to  a  large  extent,  out  of  the  environment  where  its  need  is 
recognized.  A  multi-million  dollar  facility  does  not  appear  overly  expen¬ 
sive  in  a  multi-billion  dollar  system's  acquisition  program,  but  its  cost 
does  look  sizable  when  compared  with  that  of  most  other  construction 
items  in  the  MCP.  Approval  of  a  single,  large  technical  facility  within 
the  MCP  would  necessarily  result  in  disapproval  of  many  less  costly  items; 
such  a  decision  would  undoubtedly  be  unpopular  in  view  of  the  limited 
resources  in  the  MCP.  Further,  since  the  test  facility  is  necessarily 
"justified"  by  technical  trends  and  planned  future  development  programs, 
it  usually  has  the  appearance  of  a  future  and  indefinite  need,  while  the 
other  MCP  items  demonstrate  a  more  immediate  need.  Thus,  large  test 
facilities  are  rarely  accepted  by  the  decision-makers  in  the  MCP.  The 
reason  most  often  given  for  disapproval  by  the  higher  echelons  is  that  the 
proposed  facilities  "do  not  support  an  'approved'  development  program.  " 
Such  a  rationale  has  completely  closed  the  loop  to  deny  major  facility  ac¬ 
quisition  through  either  the  systems  development  cycle  or  MCP.  This 
situation  is  briefly  explained  in  Question  20,  Appendix  B,  and  the  respond¬ 
ent  is  asked  to  suggest  methods  for  planning  and  funding  large  aeronautical 
facilities,  while  at  the  same  time  minimizing  the  risk  of  building  facili¬ 
ties  which  are  not  needed. 
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Most  of  the  respondents  were  already  aware  of  this  situation 
and  offered  a  reply.  All  but  three  acknowledge  the  seriousness  of  the 
implications.  The  author  is  aware  that  many  of  the  suggestions  have 
already  been  tried  with  little  or  no  success,  and  it  does  not  appear 
worthwhile  to  tabulate  the  responses.  However,  some  of  the  comments 
fall  into  a  few  main  themes  which  will  be  discussed. 

Many  respondents  point  out  that  the  existing  test  facilities  at 
NASA  and  AEDC  could  never  have  been  built  under  the  present  approval/ 
funding  system.  Yet,  most  major  facilities  are  still  operating  15  to  20 
years  later  and  have  contributed  significantly  to  aeronautical  successes 
in  this  country.  Hundreds  of  system  development  programs  have  passed 
through  these  facilities;  they  are  not  "one-system"  facilities.  The 
major  test  facilities  have  been  and  should  be  designed  to  provide  basic 
and  flexible  capability,  so  that  inexpensive  and  quick  modifications 
can  accommodate  new  systems'  development  needs.  Such  a  design 
philosophy  is  expected  to  minimize  the  possibility  of  designing  a 
"special  purpose"  test  facility  that  might  have  a  short-lived  need. 

There  were  numerous  suggestions  for  various  types  of  committees, 
composed  of  industry  and  government  representatives,  which  would  plan, 
endorse  and  sponsor  new  test  facilities.  It  is  likely  that  many  readers 
have  served  on  or  supported  such  committees  in  the  past.  Several 
industry  respondents  feel  that  industry  has  not  been  permitted  to  make 
sufficient  contributions  to  facility  planning  in  the  past;  they  acclaimed 
this  study  on  test  facilities  as  a  "large  step  forward.  "  However,  the 
respondents  acknowledged  that  although  committees  serve  a  useful  role 
in  coordinating  requirements  and  focusing  attention  on  the  needs,  they 
are  usually  endowed  with  only  the  power  to  make  recommendations. 

The  implementation  of  these  recommendations  often  puts  the  problem 
back  into  its  original  context. 

A  few  respondents  believe  that  the  Fitzhugh  Report's  (14)  sug- 
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gestion  for  the  creation  of  a  separate  Defense  Test  Agency  will  be 
the  solution.  They  expect  this  new  attention  at  DoD  level  to  resolve 
the  existing  approval  and  funding  problems  for  test  facilities.  Several 
respondents  expressed  a  belief  that  the  most  significant  act  to  assure 
adequate  test  facilities  and  proper  operation  of  the  facilities  would 
be  to  bring  together  the  major  government  test  facilities  (AEDC  and 
NASA)  under  one  control. 

Many  respondents  believe  that  this  situation  is  only  another 
manifestation  of  poor  DoD-AF  planning  in  general.  Perhaps  "poor  plan¬ 
ning"  is  not  correct  terminology,  for  it  implies  that  the  planners  are 
doing  a  poor  job.  This  may  not  be  true;  the  critical  problem  appears 
to  be  that  no  one  in  the  system  has  confidence  in  the  plans.  Everyone 
is  aware  that  due  to  changes  in  political  parties,  public  opinion.  Con¬ 
gressional  attitudes,  DoD  and  AF  fluctuations  in  priorities,  and  threats 
from  other  countries,  certain  portions  of  any  plan  will  be  invalid.  It 
is  concluded  by  some  government  decision-makers  that  a  substantial 
financial  investment  (like  large  test  facilities)  based  on  such  plans  is 
too  risky;  they  want  to  wait  on  a  "sure  thing.  "  It  is  this  type  of  think¬ 
ing  and  interpretation  of  plans  that  has  prevented  acquisition  of  aero¬ 
space  test  facilities  whose  need  has  been  acknowledged  by  all  the 
aerospace  industry. 


Recommendations 

1.  The  evidence  gathered  in  this  investigation  demonstrates  that  Air 
Force  policies  and  practices  have  helped  to  establish  a  competitive 
environment  which  encourages  the  aerospace  industries  to  construct 
test  facilities.  The  caliber  of  the  potential  contractor's  test  facilities 
is  influential  in  determining  his  ability  to  obtain  Air  Force  development 
contracts.  This  situation  has  resulted  in  the  proliferation  of  test  facil- 
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Ities  among  contractors;  the  capacity  for  certain  test  capability  surpasses 
the  need.  The  recent  aerospace  recession  has  amplified  the  results  of 
this  oversupply.  To  be  effective,  test  facilities  must  operate  with 
skilled  and  experienced  personnel  and  must  have  some  continuing 
improvement  and  adjustment  to  meet  changing  test  requirements.  Be¬ 
cause  of  this  rather  "fixed"  overhead,  test  costs  are  lowest  when  a 
test  facility  is  operating  at  capacity  workload.  The  proliferation  of 
test  facilities  within  industry  (and  to  some  extent  among  government 
agencies)  has  resulted  in  two  significant  trends; 

a.  The  funds  being  spent  for  new  facilities  are  too  often 
used  to  supply  redundent  test  capability,  while  the  real 
need  for  new  and  advanced  test  capability  is  going  unfilled. 

b.  The  reduced  test  workload,  when  divided  among  many 
facilities,  makes  it  uneconomical  for  each  facility  to 
maintain  the  experienced  test  crews  and  updated  test 
techniques  required  for  good  test  results. 

RECOMMENDATION;  The  Air  Force  should  review  the  usefulness  of 
incentives  which  encourage  the  contractor  to  purchase  test  facilities. 

Air  Force  policy  should  be  established  which  more  clearly  defines  the 
use  of  the  government  facilities  in  the  development  cycle  and  which 
provides  boundaries  that  better  control  the  competition  among  con¬ 
tractors  for  test  facilities.  The  contractor  should  not  be  rewarded 
for  unwarranted  duplication  of  test  capability  and  must  be  advised  that' 
he  will  be  expected  to  use  government  facilities  where  possible  in 
development  of  Air  Force  aircraft.  Until  the  problem  of  test  facility 
oversupply  is  reduced,  the  Air  Force  must  maintain  a  sufficient  work¬ 
load  at  its  own  test  facilities  to  assure  its  capacity  to  develop  new 
aircraft  and  evaluate  contractor  performance.  Some  of  the  information 
provided  in  this  report  may  prove  beneficial  to  those  who  attempt  to 
revise  AFSC  policies. 
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2.  Before  the  recent  recession  in  the  aerospace  business,  at  least 
three  aircraft  companies  were  seriously  considering  building  their 
own  high  Reynolds  number  facilities  at  costs  on  the  order  of  15-30 
million  dollars.  It  has  been  shown  that  both  industry  and  government 
personnel  expect  the  government  to  provide  the  larger  and  more  expen¬ 
sive  test  facilities;  it  was  further  demonstrated  that  this  particular 
test  need  has  been  well  identified.  Since  the  government  had  not 
been  able  to  provide  the  required  test  capability  through  its  cumber¬ 
some  and  ineffective  facility  acquisition  procedures,  the  industries 
took  the  initiative.  Certainly  the  competitive  situation  already  explain¬ 
ed  had  some  influence  on  the  industries'  decisions  to  invest  such  large 
sums  of  money  in  facilities.  However,  if  government  had  acted  promptly 
to  fulfill  its  expected  role  and  provide  the  required  capability,  in  all 
likelihood  industry  would  not  have  planned  to  take  such  large  risks.  In 
an  analogous  situation  in  the  early  1960's,  many  aircraft  companies 
built  rather  expensive  space  simulation  chambers;  few  of  the  test  facil¬ 
ities  have  been  well  utilized. 

RECOMMENDATION:  In  addition  to  discouraging  the  use  of  industries' 
test  facility  capability  in  competition  for  Air  Force  contracts,  the 
government  must  also  take  the  lead  in  constructing  new  and  advanced 
test  facilities  where  the  need  is  well  recognized.  Such  a  positive 
action  can  likely  prevent  construction  of  several  similar  facilities  at 
industrial  sites. 

3.  It  has  been  shown  that  there  are  well  identified  deficiencies  in  aero¬ 
space  test  facility  capability  and  that  these  deficiencies  are  causing  and 
will  continue  to  cause  detrimental  effects  in  systems  development.  All 
attempted  approaches  to  remedy  this  situation  have  so  far  proven  ineffec¬ 
tive.  A  decade  or  so  of  good  facility  capability  appears  to  have  resulted 
in  lack  of  a  workable  system  through  which  new  test  facilities  can  be 
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acquired. 

RECOMMENDATION;  Every  possible  avenue  should  be  tried  to  provide 
the  needed  test  capability.  Efforts  to  fund  maior  test  facilities  by 
special  committees  and  through  special  channels  should  continue.  In 
order  to  effect  a  more  permanent  solution  to  the  problem,  the  planning 
function  within  AFSC  and  USAF  should  be  revised  to  make  consideration 
of  the  supporting  test  facilities  an  integral  part  of  the  systems  plan¬ 
ning  procedure.  The  plans  should  be  designed  and  accepted  by  Air 
Force  management  as  guidelines  for  decision  and  action,  with  long 
lead-time  items  (like  maior  test  facilities)  receiving  prompt  and 
priority  attention. 

4.  The  AFSC  has  a  large  portion  of  its  capital  investment  in  test  facili¬ 
ties.  The  product  or  output  of  AFSC  is  largely  determined  by  the  use 
and  adequacy  of  these  test  facilities.  Yet  there  is  little  permanent 
expertise  within  AFSC  to  assure  that  the  facilities  are  adequately  plan¬ 
ned,  updated  and  used  properly. 

RECOMMENDATION:  It  is  suggested  that  more  expertise  in  the  facilities 
area  be  permanently  installed  in  both  the  planning  function  and  MCP 
approval  cycle  within  AFSC  (and  USAF) . 

5.  The  aerospace  experts  who  have  responded  to  this  investigation  have 
indicated  that  existing  test  facilities  can  be  made  more  useful  with  min¬ 
imal  expenditure. 

RECOMMENDATION:  AFSC  should  give  more  attention  (and  funds)  to 
improvement  in  testing  techniques,  data  interpretation  and  correlation 
between  test  and  flight  data. 

6.  It  appears  that  service  funding  at  AEDC  has  resulted  in  the  illusion 
of  a  "penalty"  for  use  of  the  AEDC  facilities  which  tends  to  reduce  the 
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effectiveness  of  the  test  facilities  and  test  personnel.  Once  Air 
Force  dollars  are  divided  into  separate  categories  (development, 
facility  operation,  etc. )  and  careers  and  jobs  are  made  to  hinge  on 
how  well  money  is  utilized  within  each  category,  it  is  all  too  easy 
to  lose  perspective  of  overall  economy  and  endanger  the  goal  of  good 
aircraft  development. 

RECOMMENDATION:  The  problems  associated  with  the  use  and  opera¬ 
tional  funding  of  AFSC  test  facilities  should  be  studied  outside  the 
influence  of  a  particular  aircraft  development  program.  In  particular, 
the  costs  of  facility  testing  should  be  studied  from  an  Air  Force  and 
national  viewpoint.  Charges  to  development  programs  for  facility  use 
at  AEDC  and  NASA  should  either  be  standardized  or  eliminated.  Service 
funding  at  AEDC  should  be  halted  to  terminate  the  very  undesirable 
side  effects  until  the  issue  can  be  evaluated  and  resolved. 
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CHAPTER  VI 

CRITIQUE  OF  FACILITY  TEST  PROGRAMS 

It  is  the  intent  of  this  portion  of  the  study  to  bring  as  much  ob¬ 
jectivity  and  expertise  as  possible  into  the  evaluation  of  the  test 
facilities'  role  and  usefulness  through  studies  of  specific  aircraft 
development  programs.  Appendix  C  contains  the  portion  of  the  ques¬ 
tionnaire  which  was  used  to  review  and  evaluate  aircraft  development 
programs  which  have  been  essentially  completed.  The  respondents  to 
these  25  questions  are  those  individuals  within  the  aircraft  companies 
who  were  closely  associated  with  the  wind  tunnel  test  programs  during 
aircraft  development  and  who  have  knowledge  of  the  relative  influence 
of  the  test  program  on  the  particular  aircraft  development  cycle.  The 
respondent  was  asked  to  provide  historical  data  on  the  wind  tunnel 
program,  to  evaluate  the  program,  and  then  to  rearrange  the  wind  tunnel 
test  program  into  a  more  optimal  and  productive  form. 

The  author  used  the  following  criteria  originally  to  select  aircraft 
development  programs  for  this  analysis: 

1.  The  portion  of  the  development  cycle  which  includes  the 
"planned"  wind  tunnel  test  program  must  have  been  completed. 
This  eliminated  some  of  the  recent  military  aircraft  presently 
under  development. 

2.  The  aircraft  must  have  undergone  a  "normal  development 
cycle.  "  This  ground  rule  was  applied  in  a  negative  sense  to 
eliminate  certain  aircraft  from  consideration.  For  example, 
aircraft  which  are  close  follow-ons  to  earlier  aircraft  would 
obviously  not  require  the  same  type  or  magnitude  of  facility 
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test  programs  as  those  which  were  brought  through  all  the 
development  stages.  Therefore,  an  attempt  was  made  to 
by-pass  those  aircraft  which  depended  heavily  upon  the 
aerodynamic  configuration  of  an  earlier  version  of  the 
aircraft. 

3.  The  more  recently  completed  aircraft  development  cycles 
were  sought;  i.  e. ,  post-1950  flight  dates. 

4.  Pure  jet-powered  aircraft  were  selected.  Propeller  aircraft 
usually  necessitate  a  different  test  program. 

5.  Representative  aircraft  from  various  categories  were  sought 

(i.e. ,  fighters,  bombers,  cargo,  light  aircraft,  military,  and 
commercial) . 

6.  A  final  and  important  criteria  was  that  the  data  had  to  be 
available  in  the  archives  of  the  aircraft  companies  and  had 
to  be  identifiable  in  the  specified  format.  Furthermore,  the 
expert  evaluator  had  to  be  available  to  contribute  to  the  study. 
And  most  important,  the  aircraft  company  had  to  be  willing  to 
provide  the  services  of  some  important  personnel  for  the 
study. 

Only  16  of  the  initially  selected  35  aircraft  and  aircraft  development 
programs  satisfy  all  of  these  criteria.  The  author  found  that  no  aircraft 
seems  to  follow  a  "normal  development  program";  every  aircraft  develop¬ 
ment  is  a  special  case  with  its  own  unique  constraints,  pressures  and 
objectives.  Needless  to  say,  criteria  Number  6  was  most  important; 
i.  e. ,  the  availability  of  the  data  and  the  evaluator,  and  the  ability  and 
willingness  of  the  company  to  participate. 

The  Boeing  Company  furnished  very  complete  data  on  the  use  of 
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the  wind  tunnel  in  their  aircraft  development  programs.  However,  these 
data  were  being  gathered  at  a  very  critical  time  in  the  Supersonic  Trans¬ 
port  (SST)  Program,  and  it  was  unfortunate  that  some  key  personnel  were 
not  available  to  evaluate  the  individual  development  programs  as  re¬ 
quested.  The  author  did  discuss  these  development  programs  in  inter¬ 
views,  and  several  of  the  Boeing  personnel  stated  that  they  were 
satisfied  with  the  use  of  the  wind  tunnel.  They  explained  that  the 
wind  tunnel  usefulness  was  so  well  accepted  at  all  levels  of  manage¬ 
ment  that  they  usually  had  no  problem  in  accomplishing  as  much  testing 
as  they  needed.  Because  of  these  circumstances,  the  actual  wind  tun¬ 
nel  program  has  been  equated  to  the  optimal  wind  tunnel  program  for  all 
the  Boeing  aircraft.  The  reader  will  observe  that  several  of  the  wind 
tunnel  programs  from  other  companies  also  did  not  change;  usually  the 
program  was  successful  and  there  was  no  reason  to  change.  Sometimes 
the  evaluator  would  reason  that  technical  knowledge  was  limited  at  that 
point  in  time  and  a  change  in  the  wind  tunnel  program  would  not  likely 
have  improved  chances  of  discovering  certain  design  deficiencies. 

The  Boeing  SST  was  included  in  these  data,  although  it  did  not 
complete  its  wind  tunnel  program.  Over  40,  000  wind  tunnel  test  hours 
had  been  accomplished  at  program  termination,  and  the  test  program  was 
projected  to  43,400  hours  at  planned  first  flight  (December  1971)  .  This 
program  was  included  because  it  represents  an  important  and  extreme 
set  of  data.  The  reader  will  find  that  the  35  aircraft  include  4  propeller 
aircraft,  3  pre-1950  aircraft,  3  aircraft  which  closely  follow  the  aero¬ 
dynamic  design  of  others,  and  one  rocket-driven  aircraft  (X-15).  The 
F-14,  which  does  meet  all  of  the  criteria,  was  not  included,  for  the 
company  did  not  wish  to  release  the  data  at  this  time. 

Usually  only  one  expert  evaluator  was  available  to  complete  the 
questionnaire  for  each  development  program.  In  the  few  cases  where 
more  than  one  estimate  was  possible,  the  data  have  been  averaged  and 
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entered  as  a  single  entry.  Despite  the  difficulties  of  obtaining  data  of 
this  type,  the  author  estimates  that  about  75%  of  the  available  data 
have  been  accumulated.  The  quantity  of  aircraft  programs  in  various 
categories  is  more  than  enough  to  prevent  conclusions  based  on  the 
data  from  being  considered  "special  cases.  "  Further,  the  reputation  of 
the  evaluators  is  so  substantial  that  their  observations  can  hardly  be 
considered  "just  another  opinion.  "  The  author  believes  that  the  opinion 
of  such  experts  based  on  independent  evaluations  of  so  many  samples 
represents  the  most  unbiased  and  truthful  data  available. 

Within  this  chapter  the  historical  data  on  the  wind  tunnel  programs 
is  presented  in  as  brief  format  as  possible.  The  evaluations  of  the  pro¬ 
grams  are  summarized  and  illustrated  with  representative  examples.  Fin¬ 
ally,  the  results  of  a  multiple  regression  analysis  are  presented  as  part 
of  a  program  to  assist  in  the  development  of  future  wind  tunnel  programs. 

As  the  respondent  is  advised  in  the  preamble  to  Question  3,  Appendix  C, 
this  portion  of  the  questionnaire  is  to  include  all  aerodynamic  or  structural 
testing  in  wind  tunnels,  but  to  exclude  development  of  the  actual  propul¬ 
sion  system.  Propulsion-airframe  integration  type  tests  are  to  be  included. 

Comparison  of  Actual  and  Optimal  Wind  Tunnel 
Test  Programs 

Table  18  lists  the  35  aircraft  which  will  be  used  in  various  combina¬ 
tions  in  the  following  discussions.  Also  designated  are  the  physical  and 
performance  characteristics  which  will  be  a  part  of  the  regression  analy¬ 
sis  (i.  e.,  weight,  thrust,  and  speed).  More  specific  definitions  are 
noted  on  the  table. 

Tables  19  through  22  are  derived  from  information  presented  in  response 
to  Questions  3  through  8,  19  and  20,  Appendix  C.  It  is  suggested  that 
the  reader  orient  himself  with  the  intent  of  the  optimal  test  program 
by  reviewing  the  preamble  to  Question  19,  Appendix  C.  Omission 
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TABLE  18 

AIRCRAFT  CHARACTERISTICS 


Takeoff 

Maximum  Total 

Maximum  Speed 

Aircraft 

Gross  Weight 

Thrust 

(MPH) 

(1,000  lbs) 

(1,  000  lbs) 

XP-84 

16.  8 

4.  9 

625 

B-47 

230.0 

36.0 

600 

F-89B 

42.  0 

14.4 

635 

B-52 

450.0 

84.0 

635 

F-lOO 

28.0 

10.  0 

770 

F-3H 

33.  9 

14.  3 

647 

F-llA 

24.  1 

10.  5 

890 

F-101 

47.0 

29,.  8 

1120 

F-8 

28.  0 

16.  0 

1000 

F-105A 

40.0 

24.  5 

1254 

KC-135 

297.0 

55.  0 

600 

B-58 

160.0 

62.4 

1385 

F-106 

35.  0 

17.  2 

1525 

707 

258.  0 

52,.  0 

600 

T-2A 

6.  9 

3.  4 

490 

F-4 

54.  6 

33.  0 

1600 

RA-5C 

62.  0 

32.0 

1385 

T-38 

11.  8 

7.  7 

860 

880 

184.  5 

44.  8 

615 

OV-1 

16.  7 

5.  5* 

325 

X-15 

36.  4 

50.0 

5500 

A- 6  A 

54.  0 

17.  0 

720 

E-2A 

49.  5 

20.  3* 

297 

727 

161.  0 

42.  0 

600 

C-141 

316.  1 

84.  0 

550 

B-70 

500.  0 

180.  0 

2000 

XC-142 

41.  5 

26.  2* 

430 

F-111 

80.  0 

38.  0 

1650 

OV-lOA 

14.  5 

3.6* 

305 

A- 7 

32.  5 

11.  4 

578 

Gulfstream  II 

56.  0 

22.  8 

585 

737 

111.  0 

28.  0 

575 

C-5 

728.0 

164.  4 

543 
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TABLE  18 — Continued 


Aircraft 


Takeoff 
Gross  Weight 
(1,000  lbs) 


Maximum  Total 
Thrust 
(1,000  lbs) 


Maximum  Speed 
(MPH) 


747 

SST 


NOTES: 

1.  Aircraft  are  listed  in  order  of  flight  date. 

2.  Maximum  total  thrust  is  maximum  static  thrust  with 
afterburner  (if  installed);  i.  e. ,  total  of  all  engines. 

3.  Maximum  speed  is  at  best  altitude. 

*  Designates  propeller-driven  aircraft.  Horsepower  is 
converted  to  "equivalent  thrust"  by  a  factor  of  2.  5  lbs.  thrust/hp 


710.0  174.0  640 

635.0  274.0  1800 


References  17,  18,  19,  20. 
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of  data  on  the  Tables  means  that  it  was  not  made  available  to  the  author. 
Although  included  on  the  Tables,  the  X-15  is  excluded  from  the  following 
analyses  because  of  the  late  arrival  of  the  data. 

Table  19  identifies  the  date  that  the  aircraft  development  program 
began.  For  the  military  aircraft,  this  is  designated  as  the  date  the 
company  was  awarded  a  contract  as  a  result  of  either  a  competitive  or 
unsolicited  proposal.  It  was  not  unusual  for  study  contracts  to  have 
preceded  this  date.  The  actual  wind  tunnel  test  hours  performed  by 
the  company  prior  to  contract  award  (or  program  go-ahead)  are  recorded 
in  the  second  column.  Sometimes  these  tests  were  conducted  as  a  part 
of  a  government-funded  study;  somtimes  they  were  company  sponsored 
as  preparation  for  their  bid  proposal.  The  third  column  designates  the 
number  of  test  hours  suggested  by  the  respondent  as  the  preferred  or 
optimal  test  program  for  the  same  point  in  time.  For  the  26  aircraft 
represented  in  the  optimal  program,  17  respondents  desired  additional 
test  hours  at  the  program  beginning,  7  saw  no  reason  to  change,  and 
2  suggested  less  testing  at  this  early  date  on  their  program.  In  all  13 
cases  where  the  difference  between  columns  two  and  three  represents 
either  a  substantial  number  of  test  hours  or  significant  portion  of  the 
total  test  program;  the  recommendation  is  for  an  increase  in  the  quantity 
of  wind  tunnel  testing  at  this  point. 

Table  20  presents  the  estimated  date  of  "design  freeze"  on  the 
aircraft  development  program.  This  is  the  date  at  which  the  wind  tunnel 
could  no  longer  make  a  significant  contribution  to  the  design  of  the 
first  flight  aircraft.  This  date  was  estimated  by  the  author  for  the  8 
Boeing  aircraft  on  the  list  (one  year  prior  to  first  flight),  and  the  actual 
test  hours  at  design  freeze  were  then  calculated  from  the  data  provided. 

The  preferred  test  program  is  again  shown  in  the  third  column.  Seven¬ 
teen  respondents  wanted  additional  testing  accomplished  at  design 
freeze  (not  necessarily  the  same  17  respondents  in  the  preceding  paragraph). 


98 


TABLE  19 

TEST  HOURS  AT  BEGINNING  OF  DEVELOPMENT  PROGRAM 

Date  Program  Actual  Test  Hours  Optimal  Test  Hours 
Aircraft _ Began _ at  Contract  Award  at  Contract  Award 


XP-84 

Jan  45 

0 

1000 

B-47 

— 

— 

— 

F-89B 

D  C  0 

45 

0 

425 

13"”  vJ  « 

F-lOO 

Nov  51 

260 

260 

F-3H 

Mar  51 

1440 

1500 

F-llA 

Dec  52 

0 

1500 

F-101 

51 

0 

1200 

F-8 

May  53 

248 

800 

F-105A 

51 

0 

2000 

KC-135 

_  _  _ 

— 

— 

B-58 

Feb  51 

0 

0 

F-106 

July  55 

0 

0 

707 

— 

— 

— 

T-2A 

June  56 

0 

250 

F-4 

Oct  54 

200 

1000 

RA-5C 

July  56 

2244 

2244 

T-38 

55 

0 

600 

880 

June  56 

0 

0 

OV-1 

Ap  57 

250 

250 

X-15 

Nov  55 

0 

0 

A- 6  A 

Feb  58 

526 

600 

E-2A 

Oct  56 

0 

1000 

727 

— 

— 

C-141 

Mar  61 

235 

1500 

B-70 

Jan  56 

1082 

3500 

XC-142 

Ap  62 

1214 

1000 

F-111 

Nov  62 

4925 

5000 

OV-lOA 

Oct  64 

430 

400 

A- 7 

Mar  64 

1634 

1634 

Gulfstream  II 

Jan  64 

0 

150 

737 

— 

— 

— 

C-5 

June  64 

1300 

2500 

747 

— 

— 

SST 

— 

— 

TEST  HOURS 


Date/Design 


Aircraft 

Freeze 

XP-84 

Oct  45 

B-47 

Dec  46 

F-89B 

47 

B-52 

Ap  51 

F-lOO 

Sept  52 

F-3H 

Mar  53 

F-llA 

Dec  53 

F-101 

July  54 

F-8 

May  54 

F-105A 

May  55 

KC-135 

Aug  55 

B-58 

Sept  54 

F-106 

Mar  56 

707 

Dec  56 

T-2A 

June  57 

F-4 

Nov  57 

RA-5C 

June  57 

T-38 

June  57 

880 

Sept  58 

OV-1 

Sept  58 

X-15 

Dec  57 

A-6A 

Dec  59 

E-2A 

Nov  59 

727 

Feb  62 

C-141 

July  63 

B-70 

June  60 

XC-142 

Feb  64 

F-111 

Nov  63 

OV-lOA 

May  65 

A- 7 

Jan  65 

Gulfstream  II 

Oct  65 

737 

Ap  66 

C-5 

May  67 

747 

Feb  69 

SST 

Dec  71 

99 


•ctual  Test  Hours 
at  Design  Freeze 


Optimal  Test  Hours 
at  Design  Freeze 


1440 

2000 

1000 

4500 

3555 

2165 

1800 

1972 

3020 

5200 

400 

5673 

1565 

1000 

374 

5014 

4579 

1822 

2069 

2005 

3977 

4476 

5605 

3700 

4282 

7955 

6712 

11785 

1110 

3657 

1124 

5400 

7000 

11800 

40000 


2000 

1625 

3555 

4500 

3500 

2500 

3200 

7000 

6000 

2100 

550 

4000 

4579 

3000 

2069 

2005 

4000 

4800 

5285 

5500 

13000 

7000 

12773 

800 

3657 

2100 

6000 


100 


5  wanted  no  change,  and  4  suggested  some  reduction  in  the  program  at 
this  point.  It  may  be  noted  that  some  of  the  changes  are  obviously 
just  round-off  approximations.  It  is  significant  that  of  the  15  changes 
that  represent  substantial  test  hours  or  a  substantial  portion  of  the 
total  test  program,  12  respondents  asked  for  an  Increase  of  testing  at 
this  point.  It  will  be  observed  that  some  of  those  who  suggested  a 
decrease  in  test  hours  for  their  program  at  design  freeze,  had  performed 
a  large  portion  of  the  test  program  at  that  time  in  the  actual  program. 

Table  21  shows  the  same  type  of  data  for  the  event  of  the  first 
flight.  This  table  represents  the  most  complete  and  exact  set  of  data. 
Date  of  first  flight  seems  to  be  well  recorded;  further,  the  actual  test 
hours  for  the  Boeing  aircraft  could  be  determined  from  the  data  provided 
the  author.  At  first  flight,  the  evaluators  suggested  14  increases  in 
wind  tunnel  test  program,  7  decreases  and  5  without  change.  For  the  11 
significant  changes  between  columns  (as  described  in  previous  para¬ 
graphs),  8  respondents  suggested  test  program  Increases.  Another  com¬ 
parison  may  be  helpful;  the  14  increases  average  1,  230  additional  test 
hours  each,  while  the  7  decreases  average  676  test  hours  each. 

The  first  two  columns  of  Table  22  designate  the  actual  and  opti¬ 
mal  total  development  test  programs.  This  is  not  meant  to  include  any 
testing  in  support  of  follow-on  versions  of  the  aircraft  or  retro-fit 
programs.  The  3  aircraft  which  Indicate  significant  overall  test  program 
increases  are  the  F-3H,  F-101,  and  B-70.  In  the  latter  case,  this 
is  the  recommended  test  program  if  development  were  directed  toward 
a  production  aircraft.  Several  aircraft  programs  show  significant  overall 
decreases  in  the  test  program,  including  the  F-89B,  XC-142,  OV-lOA  and 
C-5A.  Overall,  seven  respondents  suggested  increases  in  total  develop¬ 
ment  test  hours  and  12  suggested  decreases.  Column  3  of  Table  22 
shows  the  actual  number  of  wind  tunnel  test  hours  in  support  of  the 
development  program  after  first  flight.  A  large  number  of  test  hours  here 
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TABLE  21 


TEST  HOURS  AT  FIRST  FLIGHT 


Date  of  First 


Actual  Test  Hours  Optimal  Test  Hours 


Aircraft 

Flight 

at  First  Flight 

at  First  FJ 

XP-84 

Mar  46 

1440 

2400 

B-47 

Dec  47 

2850 

— 

F-89B 

48 

1000 

2625 

B-52 

Ap  52 

5200 

— 

F-lOO 

May  53 

4356 

4356 

F-3H 

Dec  53 

2478 

7000 

F-llA 

July  54 

2880 

4000 

F-101 

Oct  54 

2092 

3000 

F-8 

Mar  55 

5751 

4400 

F-105A 

Oct  55 

5200 

7000 

KC-135 

Aug  56 

600 

— 

B-58 

Nov  56 

8337 

8000 

F-106 

Dec  56 

2357 

2500 

707 

Dec  57 

1650 

— 

T-2A 

Jan  58 

435 

550 

F-4 

May  58 

5152 

6000 

RA-5C 

Aug  58 

7143 

7129 

T-38 

58 

4073 

4000 

880 

Jan  59 

2069 

2069 

OV-1 

Ap  59 

2165 

2165 

X-15 

Sept  59 

5400 

4786 

A-6A 

Ap  60 

4476 

5000 

E-2A 

Oct  60 

6797 

6285 

727 

Feb  63 

4075 

— 

C-141 

Dec  63 

4418 

5500 

B-70 

Sept  64 

13377 

17000 

XC-142 

Sept  64 

7213 

8000 

F-111 

Dec  64 

20587 

20040 

OV-lOA 

July  65 

1180 

900 

A- 7 

Sept  6  5 

3908 

3908 

Gulfstream  II 

Oct  66 

1969 

2150 

737 

Ap  67 

6700 

— 

C-5 

June  68 

7635 

6000 

747 

Feb  69 

14000 

— 

SST* 

Dec  71 

43400 

— 

*  Projected 
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TABLE  22 

TEST  HOURS  IN  THE  TOTAL  PROGRAM 


Actual  Total  Optimal  Total  Actual  Develop- 
Development  Development  ment  Test  Hrs 
Aircraft _ Test.  Hours  Test  Hours  After  1st  Flight 


XP-84 

— 

B-47 

— 

F-89B 

3600 

B-52 

— 

F-lOO 

— 

F-3H 

3578 

F-llA 

7632 

F-101 

2442 

F-8 

— 

F-105A 

— 

KC-135 

— 

B-58 

13765 

F-106 

2850 

707 

— 

T-2A 

632 

F-4 

6044 

RA-5C 

8627 

T-38 

4929 

880 

2472 

OV-1 

3509 

X-15 

5990 

A- 6  A 

5650 

E-2A 

8225 

727 

— 

C-141 

5476 

B-70 

13462 

XC-142 

10317 

F-111 

26041 

OV-lOA 

2360 

A- 7 

— 

Gulfstream  II 

1989 

737 

— 

C-5 

7687 

747 

— 

SST 

— 

2400 

— - 

2625 

2600 

4356 

— 

7000 

1100 

7500 

4752 

3400 

350 

9000 

— 

7000 

— 

14000 

5428 

2700 

493 

550 

197 

6000 

892 

8613 

1484 

5000 

856 

2096 

376 

3509 

1344 

4786 

— 

5000 

1174 

8225 

1428 

5500 

1058 

17000 

85 

8000 

3104 

24000 

5454 

1450 

1180 

8724 

— 

2150 

20 

6000 

52 

Actual  Total 
Testing  After 
1st  Flight 

2000 

4415 

4400 

6298 

2498 

1400 

6927 

3539 

6677 

6102 

835 

6229 

753 

5010 

8204 

2967 


1346 


18706 

4816 

2208 

1865 

2476 
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could  represent  either  correction  of  problems  discovered  in  flight  or 
design  philosophy  (i.  e. ,  where  a  significant  portion  of  the  wind  tunnel 
test  program  is  planned  in  support  of  flight  testing).  Column  4  shows 
the  total  actual  hours  spent  in  wind  tunnel  testing  after  flight  (it 
includes  the  data  of  Column  3).  The  reader's  familiarty  with  the  par¬ 
ticular  development  programs  will  make  these  data  more  meaningful. 
Sometimes  a  substantial  number  of  follow-on  wind  tunnel  hours  repre¬ 
sents  a  very  useful  and  extended  aircraft  life,  with  many  modifications 
and  adaptations  to  various  engines  and  weapons  (e.g. ,  the  B-52).  Or, 
it  could  mean  that  problems  were  discovered  during  flight  evaluation 
that  necessitated  an  unusual  amount  of  tunnel  work  after  the  first  flight 
(e.  g. ,  the  F-111). 

Evaluation  of  the  Optimal  Wind  Tunnel  Program 

The  author  had  hoped  to  make  some  correlation  of  both  the  actual 
and  optimal  wind  tunnel  test  program  with  the  data  provided  by  the 
evaluators  in  Questions  13-17,  Appendix  C.  The  respondent  is  asked 
in  Question  13  to  compare  the  development  program  being  evaluated 
with  others  in  his  experience  and  to  assess  the  level  of  technical  dif¬ 
ficulty  for  those  technical  areas  amenable  to  investigation  in  the  wind 
tunnel.  Of  the  26  aircraft  evaluated,  6  were  rated  "high,  "  19  "medium,  " 
and  1  "low.  "  The  author  observes  that  the  aircraft  that  claim  high  levels 
of  technical  advancement  in  development  did  require  more  wind  tunnel 
testing  than  their  "medium"  cousins.  The  technical  advance  which  was 
rated  "low"  was  a  direct  follow-on  and  experienced  relatively  little  wind 
tunnel  testing.  However,  within  the  range  of  "medium  technical  diffi¬ 
culty,  "  the  scatter  is  too  great  to  make  this  categorical  distinction 
worthwhile. 

Question  14  asks  for  identification  of  constraints  that  influenced 
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the  quantity  and  quality  of  the  wind  tunnel  test  program.  Many  respond¬ 
ents  noted  that  pressures  of  money  and  time  early  in  the  program  had 
a  negative  influence  on  the  wind  tunnel  test  program.  It  was  often 
stated  that  there  was  insufficient  time  to  take  data,  analyze  it,  and 
get  it  into  the  design.  These  data  are  further  substantiated  by  the 
results  of  Question  25,  Appendix  B,  which  asks  the  same  question 
without  reference  to  a  specific  aircraft.  Over  90%  of  those  respondents 
stated  that  the  wind  tunnel  programs  are  most  constrained  by  too  little 
time  to  properly  use  the  wind  tunnel  and  too  little  money  devoted  to  this 
aspect  of  the  development  program  in  the  early  stages.  The  few  other 
comments  centered  primarily  around  deficiencies  in  facility  capability 
and  problems  of  scheduling  certain  test  facilities. 

The  evaluator  is  asked,  in  Question  15,  Appendix  C,  to  rate  the 
success  of  the  particular  aircraft  development  program  in  meeting  tech¬ 
nical  objectives  in  a  timely  and  economical  manner.  Since  a  contractor 
is  likely  to  define  a  successful  program  as  one  which  results  in  the 
sale  of  many  aircraft  and  makes  a  profit,  the  respondent  is  warned  to 
ignore  the  ultimate  usefulness  or  success  of  the  system  and  evaluate 
only  the  development  program.  Of  the  aircraft  evaluated,  there  were 
no  poor  development  programs,  no  fair  development  programs,  only  4 
average  programs,  16  good  programs,  and  4  very  good  programs.  The 
author  is  aware  that  other  evaluators  outside  this  study  have  not  been 
so  kind  in  measuring  the  success  of  some  of  these  aircraft  in  meeting 
performance  objectives  in  a  timely  and  economical  manner.  In  par¬ 
ticular,  some  RAND  reports  have  provided  objective  comparisons  of 
planned  and  obtained  aircraft  flight  performance  and  development  costs 
(21).  It  would  appear  that  some  of  the  evaluators  either  did  not  under¬ 
stand  the  question  or  were  influenced  to  rate  the  success  of  their 
development  program  too  optimistically.  These  ratings  would  perhaps 
have  had  more  meaning  if  all  respondents  had  been  permitted  to  rate 
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the  "success"  of  all  development  programs;  however,  time  did  not 
permit  this  adjustment  in  procedure  after  the  final  determination  of 
the  aircraft  to  be  studied. 

Because  of  the  escalated  and  sometimes  questionable  rating  of  the 
development  project  success,  coupled  with  scattered  data  on  some 
aircraft  wind  tunnel  test  programs,  the  author  has  not  found  it  possible 
to  present  a  closed  argument  that  the  success  of  the  development  pro¬ 
gram  and  character  of  the  wind  tunnel  program  are  always  directly 
correlated.  Some  low  technical  risk  aircraft  developments  have  been 
performed  in  a  short  time  period  with  relatively  little  wind  tunnel  testing, 
and  have  been  successful.  However,  the  author  has  observed  that  for 
most  cases,  the  more  successful  development  programs  exhibit  the 
characteristic  of  a  substantial  amount  of  wind  tunnel  testing  early  in 
the  program.  Also,  the  adjustment  to  an  optimal  program  usually  re¬ 
flects  the  evaluators'  desire  for  more  wind  tunnel  testing  earlier  in  the 
development  cycle.  The  reader  has  probably  observed  from  Tables  19 
through  22  that  the  predominant  trait  of  the  optimal  wind  tunnel  programs 
is  a  shifting  of  wind  tunnel  testing  toward  the  early  stages  of  the  de¬ 
velopment  cycle.  In  fact,  many  respondents  reported  that  overall  testing 
could  have  been  reduced  if  this  had  been  practiced.  These  opinions  are 
seconded  by  the  remainder  of  the  respondents  who  answered  Question  23, 
Appendix  B.  Without  reference  to  a  particular  aircraft,  this  question 
hypothesizes  that  an  aircraft  development  program  could  be  improved  and 
cost  savings  incurred  if  more  effort  were  devoted  to  the  wind  tunnel  test 
program  earlier  in  the  development  cycle,  particularly  prior  to  design- 
freeze  on  the  first  flight  vehicle.  The  responses  are  noted  in  Table  23. 

The  hypothesis  has  very  good  acceptance  (by  84%)  with  47%  giving 
strong  agreement;  seven  per  cent  reject  the  hypothesis.  It  is  interest¬ 
ing  to  note  that  of  the  six  individuals  who  differ  with  the  large  majority, 
five  are  from  a  particular  group;  i.  e.  ,  ASD.  The  comments  of  these 
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TABLE  23 

REPLIES  TO  HYPOTHESIS  THAT  MORE  WIND  TUNNEL 
TESTING  PRIOR  TO  AIRCRAFT  DESIGN-FREEZE 
WOULD  IMPROVE  THE  DEVELOPMENT  PROGRAM 
AND  REDUCE  DEVELOPMENT  COSTS 


Agency 


Strongly  No  Agree 

Disagree  Disagree  Opinion  Agree  Strongly 


Industry  0 

ASD  1 

Headquarters  0 

AFSC  Labs  0 

AEDC  0 

NASA  0 

Other  _0 

Totals  1 


1 

4 
0 
0 
0 
0 
0 

5 


3  7  15 

2  6  5 

1  2  4 

1  7  2 

0  5  8 

0  3  5 

1_  _J_  _1. 

8  31  40 


individuals  present  the  basic  argument  that  there  is  a  trade-off  between 
optimization  of  the  aerodynamic  configuration  (which  can  be  done  more 
efficiently  in  the  wind  tunnel)  and  accumulation  of  total  system  charac¬ 
teristics  (which  can  be  done  only  through  flight  tests).  The  single 
respondent  who  objected  strongly  stated: 

"It  is  a  matter  of  degree,  obviously.  Pre-design  freeze  tests 
may  or  may  not  be  applicable;  you  really  get  down  to  working 
the  problem  after  the  design  freeze.  " 

It  appears  that  these  few  individuals  would  prefer  to  see  the  test  empha¬ 
sis  shift  toward  more  and  earlier  flight  evaluation  and  less  early  tunnel 
testing.  However,  the  overwhelming  majority  of  respondents  comment 
that  more  and  better  wind  tunnel  testing  early  in  the  development  cycle 
is  the  key  to  a  successful  development  program.  It  is  appropriate  to 
quote  the  comments  of  an  industry  respondent  who  has  years  of  direct 
experience  to  back  up  his  opinion: 

"This  type  of  testing  relates  in  a  major  way  to  the  success  or 
failure  of  the  program.  We  spend  orders  of  magnitude  more 
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"money  in  tests  to  make  a  'frozen'  design  work  than  we  spend 

in  tests  directed  to  a  true  design  optimization  process.  " 

The  reader  who  is  not  aware  of  the  importance  of  the  wind  tunnel 
data  to  aircraft  development  should  find  the  answers  to  Question  16, 
Appendix  C,  of  value.  The  expert  evaluators  were  asked  how  strongly 
the  wind  tunnel  test  program  influenced  their  particular  aircraft  develop¬ 
ment.  Twenty-two  of  the  26  replied  "very  much,  "  and  the  other  4  said 
"much.  "  The  following  comments  are  typical: 

—  The  wind  tunnel  was  essential  to  this  program. 

—  Wind  tunnel  data  were  the  basis  for  the  aerodynamic  design 
of  the  basic  airplane  and  high-lift  system.  In  many  cases, 
wind  tunnel  model  lines  were  scaled  up  and  used  directly 
for  the  full  scale  airplane.  Airloads  data  obtained  in  the 
wind  tunnel  were  used  for  structural  design.  To  a  great 
extent  the  wind  tunnel  program  paced  the  whole  development 
program  at  times;  particularly,  prior  to  configuration  freeze. 

—  Aerodynamic  design  and  airloads  data  were  totally  dependent 
on  wind  tunnel  results. 

—  This  program  was  predicated  on  developing  the  configuration 
by  wind  tunnel  tests  to  achieve  the  high  cruise  efficiency 
required.  Both  the  external  configuration  and  the  internal 
ducting  to  the  engine  were  developed  by  extensive  testing. 

The  evaluator  is  next  asked  if  he  experienced  any  difficulty  in 
obtaining  certain  data  because  of  deficiencies  in  test  facility  capability 
(Question  17,  Appendix  C).  Twenty  of  the  26  evaluators  replied  in  the 
affirmative.  However,  this  does  not  necessarily  mean  that  adequate 
facility  capability  was  unavailable;  the  author  observes  that  sometimes 
less  desirable  facilities  were  used  by  choice  (or  dictated  by  time,  money, 
availability,  etc.  constraints  noted  earlier) .  Facility  deficiencies  of 
the  1950's  and  1960's  are  presumed  to  be  of  no  particular  interest  to 
the  reader  and  will  be  omitted.  The  deficiencies  enumerated  for  modern 
development  programs  have  been  reported  in  more  detail  in  response 
to  another  question  and  are  discussed  in  the  preceding  Chapter. 
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Attention  is  now  directed  to  Questions  9  and  21  in  Appendix  C. 

The  author  was  attempting  to  derive  results  that  would  permit  definition 
of  an  optimal  test  program  by  focusing  on  portions  of  the  test  program 
to  be  conducted  in  the  various  categories  of  facilities  (as  identified 
by  charge  rate).  It  became  obvious  from  review  of  the  answers  and 
interviews  with  the  evaluators  that  many  respondents  either  could  not 
find  the  data  or  had  not  made  the  analyses  necessary  to  answer  these 
questions.  Furthermore,  the  instructions  for  calculating  charge  rates 
were  not  followed  in  all  cases.  Consequently,  a  presentation  of  the 
quantitative  replies  is  considered  useless.  The  author  has  observed 
one  trend  which  may  interest  the  reader.  A  comparison  of  the  replies 
to  Questions  9  and  21  will  demonstrate  the  evaluator's  intent  in  an 
optimal  program  to  expand  the  test  program  to  larger  and  more  costly 
facilities,  or  to  perform  less  testing  in  such  facilities.  Of  the  22  air¬ 
craft  evaluations  which  supplied  data  on  these  questions,  7  indicated 
that  a  much  larger  portion  of  the  test  load  should  be  accomplished  in 
the  more  sophisticated  facilities;  2  are  directed  toward  a  reduction 
of  testing  in  such  facilities,  and  13  imply  no  change  in  this  regard. 

It  has  been  hypothesized  that  better  use  of  the  wind  tunnel  during 
aircraft  development  would  result  in  such  benefits  as  reduced  cost, 
better  performance,  etc.  Proof  of  the  hypothesis  is  attempted  in 
Question  2  3,  Appendix  C,  through  evaluation  of  the  specific  aircraft 
programs.  The  evaluator  is  asked  if  certain  benefits  would  (in  his 
opinion)  have  been  derived  from  implementation  of  his  optimal  program. 
Recall  that  the  optimal  wind  tunnel  test  program  has  the  predominant 
traits  that  more  wind  tunnel  testing  is  accomplished  early  in  the  develop¬ 
ment  and  time  is  permitted  for  testing  and  analysis  of  data:  also,  there 
is  some  shift  toward  an  increase  in  testing  in  the  larger  and  more  sophis¬ 
ticated  wind  tunnels.  The  results  are  shown  in  Table  24. 
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TABLE  24 

IMPROVEMENTS  IN  THE  DEVELOPMENT  PROGRAM 
EXPECTED  FROM  BETTER  USE  OF  TEST  FACILITIES 

_ Question _  Yes  No 

Do  you  believe  that  implementation  of  your  hypotheti¬ 
cal  test  program  would  have  resulted  in: 

Less  overall  development  cost?  17  6 

Superior  system  performance?  17  6 

Shorter  time  to  system  demonstration?  8  15 

Less  flight  testing  required?  19  4 

At  least  one  of  the  four  benefits  was  claimed  for  each  of  the  23  aircraft 
on  which  the  data  were  furnished.  Most  experts  predict  that  the  optimal 
test  program  would  have  resulted  in  less  overall  development  costs, 
superior  performance  and  less  flight  testing.  About  one-third  would  have 
expected  a  shorter  time  period  prior  to  system  demonstration.  These 
results  are  most  significant  to  this  study.  The  respondents  explained 
these  answers  in  interview  with  the  author  based  upon  specific  problems 
associated  with  the  aircraft  development  program  being  reviewed.  The 
reduction  in  development  cost  was  primarily  expected  because  certain 
design  deficiencies  would  not  have  been  built  into  the  aircraft;  the 
respondent,  therefore,  expected  less  flight  testing  and  follow-on  wind 
tunnel  testing  to  resolve  these  deficiencies.  Respondents  pointed  out 
that  flight  testing  is  at  least  an  order  of  magnitude  more  costly  than 
wind  tunnel  testing,  and  is  not  nearly  so  effective  and  precise  for  most 
required  aerodynamic  data.  Superior  system  performance  was  expected 
because  of  a  more  optimized  design  based  on  wind  tunnel  tests.  Respond¬ 
ents  observed  that  too  often  performance  deficiencies  discovered  in  flight 
are  accepted  because  corrections  at  that  time  would  be  prohibitively 
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expensive.  Most  respondents  expected  implementation  of  the  optimal 
test  program  to  result  in  a  longer  time  period  for  development  prior  to 
flight.  However,  many  respondents  stated  in  interview  that  they  ex¬ 
pect  the  total  time  to  put  an  aircraft  in  inventory  will  not  be  so  affected 
because  of  reduced  time  spent  in  correcting  mistakes. 

It  is  desirable  to  find  specific  deficiencies  revealed  in  the  flight 
evaluation  of  the  aircraft  under  review  which  the  respondent  feels  his 
optimal  test  program  might  have  minimized  or  prevented;  the  point  is 
addressed  in  Question  24,  Appendix  C.  Four  aircraft  had  no  such 
deficiencies.  From  the  20  positive  replies,  a  few  comments  are  noted 
for  the  benefit  of  the  reader. 

—  Flaps  showed  severe  separation  from  tracks  which  were 
inadequately  simulated  on  models.  Clean  drag  was  about 
13  counts  higher  than  necessary  because  of  poor  wing- 
fuselage  fairing,  but  we  learned  too  late.  Reynolds  number 
effects  on  airloads  were  mispredicted. 

—  Engine-airframe  integration  and  high  angle-of-attack  stability 
and  control. 

--  Deficiencies  in  the  control  system  would  have  been  exposed 
with  tunnel  hinge  moment  data. 

—  Stall  characteristics.  More  tunnel  time  at  an  earlier  date 
could  have  precluded  some  flight  tests  and  later  wind  tunnel 
tests. 

--  Changes  in  wing  camber  made  after  flight  could  have  been 
determined  in  earlier  wind  tunnel  tests. 

--  Buffet  surveys  in  the  wind  tunnel  would  have  resulted  in 
aerodynamic  fix  and  better  performance. 

—  Horizontal  tail  hinge  line  was  relocated  as  a  result  of  flight 
testing.  Leading  edge  slat  was  found  unnecessary  as  a 
result  of  flight  testing. 

—  Tail  hinge  moment  problem,  rudder  size  for  spin  recovery,  and 
flap  buffet. 
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One  respondent  who  answered  "no"  made  this  interesting 
comment: 


—  However,  much  redesign  and  schedule  slippage  pressure  could 
have  been  avoided  if  the  optimal  wind  tunnel  test  program 
could  have  been  followed.  Obviously,  many  millions  of 
dollars  of  cost  could  have  been  saved  as  well. 

This  same  theme  is  pursued  in  Question  24,  Appendix  B,  for  the 
remainder  of  the  respondents.  It  is  speculated  that  some  deficiencies 
found  in  the  flight  evaluation  phase  might  have  been  prevented  with  a 
more  thorough  facility  test  program,  but  with  perhaps  attendant  increases 
in  time  and  cost  of  that  phase.  The  respondent  is  asked  to  discuss  the 
results  of  his  experiences  on  the  relative  costs  of  facility  testing  versus 
correction  of  design  deficiencies,  and  express  any  ideas  he  may  have 
to  optimize  the  trade-off.  Most  of  the  respondents  answered  this  ques¬ 
tion  with  general  comments  that  more  early  wind  tunnel  testing  was 
desirable,  that  wind  tunnel  testing  was  much  cheaper  and  desirable  for 
certain  data,  etc.;  i.  e.,  the  same  arguments  that  have  already  been 
presented.  A  few  respondents  attempted  to  quantify  their  answers;  two 
of  these  comments  are  noted  below. 

From  an  ASD  respondent: 

I  believe  that  80  to  90  per  cent  of  the  cost  required  to  fix  up 
a  new  aircraft  after  flight  tests  have  started  could  be  saved 
by  running  more  wind  tunnel  tests  before  final  design  freeze. 

This  is  based  on  our  recent  F-111  experience.  This  is  probably 
true  for  any  subsonic-supersonic  configuration  where  design 
compromises  are  often  required  for  efficient  supersonic  flight. 

From  an  AFSC  Laboratory  respondent: 

The  cost  of  "fixing  the  fleet"  is  several  orders  of  magnitude 
greater  than  an  adequate  test  program.  Based  on  a  risk  analy¬ 
sis  study,  as  much  as  one-half  the  "get  well"  costs  should 
be  spent  on  a  test  program. 
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A  few  respondents  have  challenged  the  author  to  prove  that 
certain  deficiencies  found  on  particular  aircraft  could  have  been  found 
in  wind  tunnel  tests  and  prevented.  Unfortunately,  it  is  impossible  to 
"prove"  that  testing  which  was  not  accomplished  would  have  resulted 
in  any  specific  results.  However,  the  overwhelming  agreement  and 
testimony  of  many  of  this  nation's  most  knowledgeable  experts  should 
convince  the  skeptic  that  the  wind  tunnel  test  programs  are  not  in 
proper  balance. 

The  argument  presented  here  is  not  an  either/or  case;  i.  e. ,  flight 
testing  or  facility  testing.  Certainly  there  is  no  claim  that  all  problems 
can  be  found  in  the  wind  tunnel.  Certain  interface  problems  and  other 
unknowns  or  unexpected  problems  will  often  appear  in  flight  evaluation. 
Several  respondents  (primarily  from  government  circles)  have  observed 
that  the  "major  problems"  always  occur  in  flight  tests.  These  are  all 
"truths";  they  do  not,  however,  lead  to  the  conclusion  that  one  should 
push  for  early  prototype  aircraft  at  the  sacrifice  of  an  adequate  facility 
test  program.  Certainly  the  "major  problems"  are  discovered  in  flight; 
discovery  at  this  development  stage  makes  them  major  problems.  Cor¬ 
rections  made  in  the  wind  tunnel  are  inexpensive  and  never  make  the 
headlines.  The  message  that  comes  from  the  large  majority  of  expert 
respondents  is  this: 

The  present  balance  between  preventing  problems  and  solving 
problems  is  far  from  adequate.  Industry's  and  government's 
backlog  of  experience  is  highly  oriented  toward  fixing  prob¬ 
lems  which  may  result  from  insufficient  testing  and  develop¬ 
ment.  This  is  being  forced  by  reluctance  to  commit  appropriate 
funds  at  a  rate  sufficient  to  support  an  optimal  program  plan. 
Facility  testing  cannot  solve  all  the  problems;  however,  it 
can  greatly  reduce  the  possibility  of  problems. 
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Functions  of  the  Wind  Tunnel  in  a  Development  Program 

The  wind  tunnel  is  used  to  perform  a  variety  of  functions  in  the 
aircraft  development  program.  The  priority  attached  to  these  functions 
has  some  influence  on  the  use  of  the  wind  tunnel  and  its  contributions 
to  development.  It  is  interesting  to  observe  how  different  groups  of 
respondents  rate  the  purposes  for  testing  in  the  wind  tunnel.  Question 
27,  Appendix  B,  seeks  to  determine  the  respondents'  ranking  of  importance 
of  functions  which  denote  categories  of  use  of  the  wind  tunnel  in  the 
development  program.  The  results  are  shown  in  Table  25.  The  functions 
are  defined  as  follows: 

a.  Verify  the  design  hypotheses;  i.  e.  ,  assure  that  theory 
and  past  experiences  have  been  applied  correctly  and  pro¬ 
duce  expected  results. 

b.  Generate  new  design  information  and  improvement  in  the 
design  concept;  i.  e. ,  produce  new  technology  to  be  applied 
to  the  system. 

c.  Expose  difficulties  which  may  have  been  overlooked;  i.  e.  , 
look  for  undefined  and  unexpected  problems. 

Overall,  the  respondents  choose  category  "a"  as  first  choice  preference, 
"c"  as  second  choice  preference,  and  "b"  as  third  choice  preference. 

The  NASA  respondents  are  more  inclined  to  argue  the  trend.  This  re¬ 
sponse  might  be  expected  because  of  their  strong  research  orientation; 
i.  e. ,  they  could  be  expected  to  rate  option  "b"  higher  than  others. 

Some  readers  have  probably  already  observed  that  a  different  conclusion 
may  be  deducted  from  these  results.  If  one  looks  only  at  first  choices, 
option  "b"  ranks  second  instead  of  third.  Note  the  solidarity  of  the 
ASD  response.  Some  ASD  respondents  commented  that  option  "b"  should 
not  even  be  included  in  a  development  program;  i.  e. ,  they  feel  that 
production  of  new  technology  is  not  part  of  systems  development.  Other 
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groups  of  respondents  do  not  tend  to  feel  so  strongly  as  those  from  ASD. 


TABLE  25 

RESPONDENTS'  PRIORITY  FOR  FUNCTIONS 
OF  WIND  TUNNEL  TESTING 


Agency 

Number  of  1st 
Choices 

Number  of  2nd 
Choices 

Number  of  3rd 
Choices 

No 

Answer 

a 

b 

c 

a 

b 

c 

a 

b 

c 

Industry 

11 

7 

3 

7 

5 

9 

3 

9 

9  . 

7 

ASD 

14 

3 

1 

3 

1 

14 

1 

14 

3 

0 

Hdqtrs. 

4 

0 

2 

1 

3 

2 

1 

3 

'  2 

1 

AFSC  Labs 

4 

2 

4 

4 

1 

5 

2 

7 

1 

0 

AEDC 

7 

2 

1 

2 

1 

7 

1 

7 

2 

3 

NASA 

1 

3 

3 

4 

2 

1 

2 

2 

3 

1 

Other 

0 

1 

1 

1 

1 

0 

1 

0 

1 

1 

Totals 

41 

18 

15 

22 

14 

38 

11 

42 

21 

13 

The  results  of  Table  25  do  not  include  the  responses  of  the  evalua¬ 
tors  of  the  specific  aircraft  programs.  These  respondents  were  asked 
in  Question  25,  Appendix  C,  to  designate  the  relative  importance  of 
the  three  categories,  both  in  the  aircraft  development  program  being 
evaluated  and  in  the  optimal  development  program.  It  was  intended  to 
determine  how  the  optimal  use  of  the  wind  tunnel  differs  from  the  actual 
use.  The  results  are  shown  in  Table  26. 

Although  the  overall  trend  of  the  results  in  Table  26  is  the  same 
as  that  exhibited  in  Table  25,  the  move  toward  an  optimal  test  program 
indicates  a  slightly  different  emphasis.  In  Table  26,  strong  stratifica¬ 
tion  of  functional  emphasis  is  noted  for  the  actual  programs  evaluated 
(similar  to  that  demonstrated  by  the  ASD  responses  in  Table  2  5).  How¬ 
ever,  a  moderating  trend,  with  more  emphasis  on  generation  of  new 
design  information,  is  observed  in  the  categories  of  the  optimal  programs. 
A  few  respondents  have  added  a  fourth  wind  tunnel  test  function  to  the 
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list;  i.  e. ,  support  of  flight  test  programs.  However,  most  ranked  it 
fourth  in  priority. 


TABLE  26 

COMPARISON  OF  WIND  TUNNEL  FUNCTIONAL 
PRIORITIES  FOR  ACTUAL  AND  OPTIMAL  WIND 
TUNNEL  TEST  PROGRAMS 


Actual  Programs  Evaluated 


No.  of  First  Choices 


No.  of  Second  Choices 


No.  of  Third  Choices 


23 


21 


18 


Optimal  Programs 


16  8  6  8 


8  14  6 


14  10 


The  only  safe  conclusion  to  be  drawn  from  these  data  is  that  there 
are  a  variety  of  opinions  regarding  the  priority  of  the  functions  of  wind  tun¬ 
nel  testing.  However,  there  may  be  some  significance  to  the  observa¬ 
tion  that  the  ASD  reponses  are  consistent  with  the  estimates  of  priorities 
in  the  actual  development  programs.  This  could  mean  that  ASD  has 
exerted  a  significant  influence  on  the  type  and  timing  of  wind  tunnel  test¬ 
ing  in  the  actual  development  programs.  If  this  be  the  case,  it  should 
be  observed  that  the  recommended  optimal  test  programs  reflect  some 
change  in  emphasis;  particularly,  there  is  more  recognition  of  the  wind 
tunnel's  ability  to  generate  new  design  information  and  produce  improve¬ 
ments  in  design  concepts  during  development. 
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Design  of  an  Optimal  Wind  Tunnel  Program 

There  are  many  occasions  when  the  aerospace  engineer  or  manager 
has  need  for  an  estimating  procedure  which  predicts  the  magnitude  and 
type  of  wind  tunnel  test  program  which  would  be  appropriate  for  a  pro¬ 
posed  new  aircraft  system.  Such  occasions  include  the  potential  con¬ 
tractor's  estimate  for  his  proposal,  the  government  evaluator's  review 
and  comparison  of  contractor's  proposals,  the  planner's  estimate  of 
future  development  program  cost  and  time,  or  perhaps  the  wind  tunnel 
operator's  estimate  of  future  workload.  The  development  of  the  new  wind 
tunnel  program  usually  involves  a  lengthy  review  and  evaluation  of  past 
wind  tunnel  programs  to  provide  a  basis  for  extrapolation.  It  is  the  pur¬ 
pose  of  the  following  analyses  to  develop  a  method  to  permit  an  estimate 
of  an  optimal  wind  tunnel  test  program  with  limited  knowledge  about  the 
future  aircraft  program  and  without  the  review  of  past  programs.  Such  an 
estimating  procedure  should  serve  as  a  means  for  determining  quick  and 
approximate  wind  tunnel  programs  and  should  provide  a  basis  on  which  to 
build  more  detailed  test  programs. 

It  is  common  to  observe  in  the  literature  trend  lines  which  show 
wind  tunnel  test  hours  on  aircraft  development  programs  as  a  function  of 
development  or  flight  year  (22) .  The  author  has  observed  that  many  of 
the  aerospace  engineers  associated  with  wind  tunnel  programs  have  their 
own  private  collection  of  such  data.  Depending  upon  the  time  frame  of 
the  data  and  the  aircraft  selected  for  Inclusion,  the  originator  of  such  a 
plot  can  demonstrate  various  trends  of  his  own  choosing.  Several  things 
about  this  practice  have  concerned  the  author.  First,  some  of  the  data 
points  on  such  plots  usually  represent  very  poorly  designed  wind  tunnel 
test  programs;  i.  e. ,  data  points  which  represent  either  too  little  or  too 
much  testing  are  used  as  a  basis  for  extrapolation.  Then,  too,  the 
author  has  observed  that  sometimes  the  data  are  not  consistent  in  that 


117 


they  may  represent  either  the  quantity  of  testing  at  first  flight  or  total 
wind  tunnel  testlng(which  often  includes  many  modification  and  retro¬ 
fit  programs).  A  third  concern  is  the  wide  scatter  in  the  data,  which 
is  evident  even  when  one  tries  to  camouflage  the  dispersion  by  starting 
with  very  old  data  and  congregating  the  modern  data  on  a  semi-log  plot. 

Some  of  these  curves,  which  are  sharply  rising  exponentials,  appear  to 
provide  a  poor  basis  for  extrapolation.  All  the  trends  demonstrate  some 
increase  in  amount  of  testing  with  increase  in  calendar  years.  The  need 
for  some  quick  and  simple  estimating  procedure  is  demonstrated  by  the 
fact  that  such  calendar  year-test  hour  trend  lines  are  used  for  making 
predictions;  i.  e. ,  it  is  insinuated  that  a  proposed  aircraft  development 
program  should  have  a  certain  quantity  of  testing  based  upon  extrapola¬ 
tion  of  such  curves.  The  method  for  estimating  the  magnitude  of  a  wind 
tunnel  program  to  be  developed  here  will,  hopefully,  provide  some  cor¬ 
rection  of  the  faults  noted  above. 

There  is,  indeed,  some  positive  correlation  between  wind  tunnel 
test  hours  and  calendar  year  of  first  flight.  However,  if  one  will  closely 
observe  the  characteristics  of  the  aircraft  whose  test  programs  have  been 
reported,  he  will  be  able  to  make  a  few  generalizations  about  the  data. 

For  a  given  time  period,  those  aircraft  that  have  had  the  most  testing  in 
a  wind  tunnel  are  usually  either  faster,  bigger,  heavier,  or  have  more  thrust 
than  those  which  have  had  less  testing.  Such  observations  led  the 
author  to  conclude  that  multiple  predictors  might  provide  a  better  estimate 
of  wind  tunnel  test  hours  than  calendar  year  alone.  One  would  expect 
the  systems  to  require  more  testing  with  increasing  time  as  they  become 
more  complex  and  push  technical  limits  more  closely.  However,  one 
must  acknowledge  that  accumulated  technology  should  counter  this  trend 
and  tend  to  reduce  the  slope  of  the  curve  as  time  increases.  Thus,  some 
of  the  physical  and  performance  characteristics  of  the  aircraft  will  be 
added  to  the  time  factor  in  a  multiple  regression  analysis  in  an  attempt 
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to  provide  a  better  correlation. 

The  idea  of  the  optimal  test  program  was  originated  to  eliminate 
inadequate  test  programs  from  the  basic  data.  The  "experts,  "  who  know 
the  most  about  the  particular  aircraft  development  program  and  the  use¬ 
fulness  of  the  data,  have  carefully  reviewed  the  original  test  programs 
and  made  the  adjustments  which  they  believe  would  have  made  them 
more  effective.  The  characteristics  of  the  optimal  test  program  have 
already  been  observed.  Certainly,  more  individual  evaluations  on  each 
aircraft  test  program  optimization  would  have  been  desirable  to  reduce 
the  influence  of  any  one  man's  opinion;  as  noted,  this  was  not  usually 
possible.  The  author  had  also  hoped  to  include  more  aircraft  in  his  sur¬ 
vey;  for  reasons  stated  earlier,  this,  too,  was  impossible. 

A  standard  multiple  regression  computer  program  was  used  to  test 
the  ability  of  a  number  of  independent  variables  to  predict  optimal  test 
hours  at  first  flight.  The, computer  program  included  a  sub-routine  which 
permitted  expression  of  non-linear  variables  in  linear  form.  The  author 
took  advantage  of  this  program  to  test  various  logarithmic  and  natural 
combinations  of  all  variables.  The  various  predicting  equations  were 
evaluated  by  observation  of  the  multiple  correlation  coefficient  and  a 
study  of  the  individual  residuals  (i.  e. ,  optimal  test  hours  minus  predicted 
test  hours}.  The  significance  of  the  predictors  (in  various  combinations) 
was  determined  with  an  F-test.  The  reader  is  directed  to  Appendix  E  for 
a  more  complete  description  of  the  regression  program,  mathematical 
expressions,  and  results  that  are  not  detailed  in  the  text. 

One  must  exercise  care  in  the  interpretation  of  a  correlation  coef¬ 
ficient  as  a  measure  of  the  linear  relationship  between  variables.  It 
is  simply  a  mathematical  interpretation  and  is  completely  devoid  of  any 
cause  or  effect  Implications.  One  must  use  his  familiarity  with  the 
variables  and  the  field  of  application  to  determine  if  the  variables  have 
any  effect  on  each  other.  The  author  has  already  expressed  his  concern 


119 


over  the  use  of  calendar  year  as  a  single  predictor.  However,  it  is  the 
author's  opinion  that  speed,  weight  and  thrust  should  rise  and  fall  in 
concert  with  test  hours.  Certainly  the  faster  aircraft  require  more  test¬ 
ing,  simply  because  they  must  be  evaluated  in  several  speed  regimes 
and  in  several  types  of  tunnels.  As  aircraft  weight  Increases,  the  size 
and  cost  of  the  aircraft  usually  increases  and  interest  in  reducing  risk 
increases.  As  total  thrust  increases,  either  aircraft  size  or  performance 
(or  both)  increase;  either  implies  more  testing  in  development.  It  is 
suggested  that  calendar  year  is  only  indirectly  associated  with  quantity 
of  testing  through  its  correlation  with  these  and/or  other  variables. 

The  first  case  tried  included  33  aircraft.  These  are  all  that  are 
listed  in  Table  18  except  the  X-15  and  XC-142,  which  the  author  felt 
were  too  unique  to  be  compatible  with  the  other  data.  Four  regression 
analyses  were  performed;  i.  e.  ,  both  dependent  and  independent  variables 
were  used  in  both  logarithmic  and  natural  (raw)  format.  For  example,  one 
combination  would  be: 

Log  optimal  hours  and  raw  predictors. 

The  "raw  data"  are  the  natural  numbers.  The  reader  will  observe  several 
possible  "curve"  shapes  from  these  expressions.  The  predictors  which 
were  tested  were: 

Weight 

Thrust 

Speed 

Calendar  Year  of  First  Flight 
Categories: 

Combat/Non-combat 
Fight  er/Bomber 
Miscellaneous/ Cargo 

The  first  three  predictors  are  as  described  in  Table  18.  The  "categories" 
can  be  considered  additional  independent  variables.  It  may  be  observed 
in  Appendix  E  that  none  of  the  trials  with  logarithmic  expressions  gave 
as  good  correlation  as  the  one  with  the  raw  data. 
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Calendar  year  of  first  flight  was  used  as  a  single  predictor  to  give 
a  correlation  coefficient  (R)  of  0.  506.  A  combination  of  weight,  thrust 
and  speed  gave  a  multiple  correlation  coefficient  of  0.918  (1.000  is 
perfect  correlation).  Observation  of  the  F-ratios  shows  that  neither  year 
nor  the  categories  add  to  the  significance  of  the  prediction.  Unfortunately, 
a  close  examination  of  the  results  showed  that  the  correlation  statistics 
were  somewhat  misleading  because  one  aircraft  (the  SST)  had  such  a 
large  influence  on  the  data. 

The  decision  was  made  to  eliminate  from  consideration  a  number  of 
aircraft  for  various  reasons  and  try  another  regression.  The  older  aircraft 
were  omitted  (the  F-84,  F-89,  and  B-47  have  flight  dates  prior  to  1950); 
the  three  propeller  aircraft  were  omitted  (the  OV-1,  E-2A,  and  OV-lOA); 
the  smaller  trainer  aircraft  were  eliminated  (T-2A  and  T-38);  the  F-106, 
KC-135,  and  707  have  reduced  development  cycles  and  were  omitted  (the 
F-106  closely  followed  the  F-102,  and  the  707  followed  the  KC-135,  which 
itself  followed  the  Boeing  367-80);  the  SST  was  omitted  both  because  of 
its  dominance  over  the  data  and  its  irregular  development  cycle  (three 
incomplete  aerodynamic  configuration  developments);  and  the  F-111  was 
eliminated  because  of  its  variable-sweep  wing.  The  variable-sweep 
wing  requires  more  wind  tunnel  testing  because  of  the  many  configura¬ 
tions;  i.  e.  ,  essentially,  more  than  one  aircraft  is  being  developed. 

The  20  surviving  aircraft  were  used  in  a  multiple  regression  similar 
to  the  preceding  case.  This  time,  the  categories  were  dropped  as  in¬ 
dependent  variables.  Again  the  raw  dependent  and  independent  variables 
showed  better  correlation  than  the  various  logarithmic  combinations.  Thrust/ 
weight  ratio  was  tried  as  an  additional  predictor  (independent  variable) 
to  see  if  it  would  aid  the  raw-raw  correlations.  It  may  be  observed 
from  the  tabulated  data  in  Appendix  E  that  neither  thrust/weight  ratio 
nor  calendar  year  add  significantly  to  any  of  the  better  combinations  of 


predictors.  For  this  case,  calendar  year  of  first  flight  has  a  correlation 
coefficient  of  0.  317  when  used  alone.  The  better  predictors  and  their 
multiple  correlation  coefficients  are: 
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W=  takeoff  gross  weight  (pounds) 

S=  maximum  speed  at  best  altitude  (miles  per  hours). 

This  prediction  equation  has  a  reasonably  high  multiple  correlation  co¬ 
efficient  for  the  20  subject  aircraft  (R  =  0.  856);  the  constant  is  not 
unreasonably  large;  there  are  three  predictors  which  intuitively  corre¬ 
late  with  the  dependent  variable;  the  residuals  do  not  have  any  highly 
irregular  pattern;  and  it  appears  to  predict  adequately  outside  its  range 
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of  variables.  It  may  be  observed  in  Appendix  E  that  the  predicting  equation 
preferred  by  the  author  does  not  give  the  largest  F-ratio.  However,  the 
rationale  for  this  choice  is  further  explained  in  the  Appendix.  The  reader 
will  observe  that  other  predicting  equations  based  on  different  variables 
and  combination  of  variables  are  also  given  in  Appendix  E.  These  will 
prove  useful  if  one  is  not  provided  sufficient  predictors  to  use  the  prefer¬ 
red  equation. 

The  reader  may  share  the  author's  earlier  concern  that  the  "weight" 
term  in  the  preferred  predicting  equation  is  negative,  although  it  is  shown 
in  Appendix  E  that  weight  used  as  a  single  predictor  is  positively  corre¬ 
lated  with  optimal  test  hours.  It  can  be  illustrated,  by  working  out 
several  examples,  that  thrust  is  the  significant  determinant  of  test  hours 
for  most  of  the  aircraft;  further,  there  is  a  very  good  correlation  between 
thrust  and  weight  (this  information  is  not  presented).  It  was  observed 
that  the  "weight"  term  was  negative  in  all  the  equations  which  included 
tho  "thrust"  term,  both  in  Appendix  E  and  in  the  many  examples  not  in¬ 
cluded  in  the  presented  information.  The  author  has  concluded  that  there 
is  no  significance  to  the  negative  "weight"  term;  this  is  simply  the  solu¬ 
tion  to  the  regression  equation  for  the  data  presented.  It  does  not  mean 
that  higher  weight  aircraft  require  less  testing;  it  does  mean  that 
weight,  in  concert  with  the  other  predictors  in  the  equation,  best  describes 
the  optimal  test  programs  for  the  given  aircraft  in  the  manner  illustrated. 

The  test  hours  at  first  flight,  as  predicted  by  the  preferred  equation, 
are  shown  for  the  20  aircraft  in  Table  2  7.  Also  shown  are  the  residuals; 
i.  e.  ,  optimal  test  hours  at  first  flight  minus  the  predicted  optimal.  Fairly 
large  residuals  were  expected  because  the  data  reflect  the  test  philosophies 
of  both  the  evaluators  and  the  aircraft  companies.  Historically,  some 
companies  perform  more  testing  than  others.  The  predicted  test  hours 
represent  an  "averaged"  value  of  expert  opinion  and  test  philosophy.  For 
example,  when  two  aircraft  are  as  close  in  performance,  size  and  time 
period  as  the  C-5A  and  747,  yet  are  subject  to  such  widely  divergent 
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TABLE  27 

PREDICTED  TEST  HOURS  AT  FIRST  FLIGHT 
20  AIRCRAFT 


Aircraft 


Predicted  Test  Hours  Residual 

at  First  Flight _ 


F-llA 

3942 

58 

Gulfstream  II 

4456 

-  2306 

A- 6  A 

3967 

1033 

F-105 

5730 

1270 

FvIOl 

6066 

-  3066 

F-3H 

3922 

3078 

F-4 

6872 

871 

RA-5C 

6403 

726 

880 

4754 

-  2685 

B-70 

16586 

414 

F-lOO 

3672 

684 

C-141 

6910 

-  1411 

C-5A 

8908 

-  2908 

A- 7 

3525 

383 

F-8 

4648 

-  248 

B-58 

8173 

-  173 

B-52 

4582 

617 

727 

4832 

-  757 

737 

4060 

-  2640 

74  7 

10479 

3521 

Note:  Predicted  Test  Hours  =  function  (weight,  thrust,  speed). 
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estimates  of  optimal  test  hours,  large  residuals  are  assured.  Note  that 
the  regression  equation  brings  the  C-5A  test  program  more  in  line  with 
its  actual  program  and  reduces  the  optimal  program  for  the  747.  The 
reader  who  is  familiar  with  the  individual  aircraft  development  programs 
can  explain  many  of  the  variations  between  the  actual  and  predicted 
test  hours.  For  example,  consider  the  F-101,  whose  optimal  test  hours 
at  first  flight  are  over- predicted  by  the  regression  equation  by  a  factor 
of  two.  One  must  account  for  the  fact  that  the  F-101  was  the  successor 
to  the  XF-88,  an  experimental  aircraft,  and  therefore  had  advantage  of 
a  substantial  quantity  of  data  on  the  basic  aerodynamic  configuration. 
The  predicted  test  program  is  in  keeping  with  a  developmental  program 
that  starts  without  such  a  substantial  backlog  of  data.  On  the  other 
hand,  the  F-3H  is  under-predicted  by  a  substantial  margin.  On  Table 
21  it  is  observed  that  this  particular  evaluator  almost  tripled  "actual 
test  hours"  for  his  "optimal  test  hours"  estimate;  this  is  by  far  the 
largest  per  cent  change  in  any  program  evaluated.  The  "averaged"  esti¬ 
mate,  as  predicted  by  the  regression  equation,  shows  that  the  optimal 
test  program  should  have  a  substantial  increase,  but  not  so  large. 

The  regression  equation  based  on  the  20  aircraft  has  also  been 
used  to  predict  the  optimal  test  hours  at  first  flight  for  those  aircraft 
omitted  from  the  regression  (the  results  are  shown  in  Table  28).  The 
older  programs  fit  nicely,  but  the  propeller  aircraft  are  not  predicted  so 
well.  Those  programs  which  were  identified  as  close  follow-ons  to 
others  (F-106,  KC-135,  and  707)  are  reasonably  predicted,  if  one  adds 
cumulative  hours  from  preceding  development  programs.  The  equation 
is  not  recommended  for  vertical  or  short -takeoff  aircraft;  neither  does 
it  predict  rocket  driven  aircraft  test  programs. 

The  regression  equation  predicts  "averaged"  or  "standard"  test 
programs;  one  must  adjust  the  prediction  to  fit  the  needs  of  the  particu¬ 
lar  aircraft.  The  author  has  considered  the  predictions  for  the  F-111 
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TABLE  28 

PREDICTED  TEST  HOURS  AT  FIRST  FLIGHT 
OUTSIDE  THE  REGRESSION  DATA 


Aircraft 

Predicted  Test  Hours 
at  First  Flight 

Residual 

XP-84 

3100 

-  700 

F-89 

3770 

-  1145 

B-47 

2870 

20 

OV-1 

2825 

-  660 

E-2A 

3910 

2375 

OV-lOA 

2620 

-  1720 

T-2A 

2945 

-  1395 

T-38 

3800 

200 

F-106 

5270 

-  2770 

KC-135 

3880 

-  3280 

707 

4240 

-  2590 

F-111 

7000 

13040 

SST 

25000 

18400 

XC-142 

5000 

3000 

X-15 

13775 

-  8989 

Note:  Predicted  Test  Hours  =  function  (weight,  thrust,  speed). 
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and  SST  in  Table  28  and  reviewed  the  test  programs  for  more  modern 
military  aircraft  not  included  in  this  study  (F-14,  F-15  and  B-1).  Some 
corrections  are  suggested  for  use  of  the  regression  equation.  Aircraft 
that  incorporate  novel  aerodynamic  design  which  is  of  a  "pioneering" 
nature  should  have  an  increase  in  test  hours  by  a  factor  of  about  1.  5. 

The  same  is  true  for  aircraft  which  extend  the  performance  envelope 
beyond  standard  technology,  or  strive  for  operational  performance  at 
several  extreme  operating  conditions.  Any  of  these  events  should  in¬ 
crease  the  amount  of  testing  required.  The  swing-wing  aircraft  "pre¬ 
dicted  test  program"  should  be  adjusted  upward  by  a  factor  of  two 
(after  the  corrections  noted  above  are  made).  Of  course,  these  adjust¬ 
ment  factors  must  be  moderated  as  certain  technologies  become  more 
standard. 

There  are  additional  guidelines  for  establishing  an  optimal  test 
program  which  can  be  derived  from  the  available  data.  Tables  29  and  30 
provide  estimates  for  the  subdivision  of  optimal  wind  tunnel  test  hours 
for  transonic  and  supersonic  aircraft  by  wind  tunnel  type  (i.  e. ,  subsonic, 
transonic,  and  supersonic).  Spin  testing  is  included  in  subsonic  test¬ 
ing.  These  data  are  based  on  the  optimal  test  program  as  predicted  by 
the  evaluator  where  the  information  is  available;  otherwise,  they  are 
based  on  the  actual  development  program  or  the  actual  total  test  program. 
One  may  find  estimates  for  transonic  aircraft  in  Table  29  and  for  super¬ 
sonic  aircraft  in  Table  30.  Of  course,  the  suggested  test  program  should 
be  further  adjusted  to  reflect  emphasis  in  test  regimes  where  problems 
are  known  or  suspected. 

Table  31  permits  one  to  phase  the  test  program  with  time.  As  in 
all  these  techniques  for  prediction,  the  average  opinion  is  suggested. 
However,  it  is  observed  that  the  overall  average  for  all  24  aircraft  and 
the  average  for  the  various  categories  shows  little  variation.  The  optimal 
test  hours  at  first  flight,  as  predicted  by  the  regression  equation,  represent 
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TABLE  29 

SUBDIVISION  OF  TEST  PROGRAMS  BY  WIND  TUNNEL  TYPE: 
TRANSONIC  AIRCRAFT 

%  Subsonic  %  Transonic 


Aircraft _ Testing _ Testing 


Fighter/Attack 

F-lOO 

35 

65 

Note:  These  data  are 

F-3H 

36 

64 

based  on  the  optimal 

F-llA 

67 

33 

development  test 

F-8 

39 

61 

program  except  as 

A-6A 

74 

26 

designated. 

A-7 

53 

47 

Average 

51 

49 

*  Based  on  total  amount 

of  testing. 

Bomber 

B-52 

77 

23  * 

Supersonic  testing  for 
the  F-lOO,  F-3H,  F-llA 

Large  Commercial 

and  F-8  is  included  in 

707 

52 

48  * 

transonic  testing. 

880 

72 

28 

727 

57 

43  * 

737 

41 

59  * 

747 

53 

47  * 

Average 

55 

45  * 

Military  Cargo 

C-141 

63 

37 

C-5 

58 

42 

Average 

60 

40 

Small  Commercial 

Gulfstream  II 

84 

16 

Trainer 


T-38 


70 


30 


TABLE  30 


SUBDIVISION  OF  TEST  PROGRAMS  BY  WIND  TUNNEL  TYPE: 
SUPERSONIC  AIRCRAFT 


Aircraft 

%  Subsonic 
Testing 

%  Transonic 
Testing 

%  Supersonic 
Testing 

Fighter/Attack 

F-101 

35 

35 

30 

F-105 

43 

31 

26 

F-106 

30 

35 

35 

F-4 

25 

42 

33 

RA-5C 

41 

15 

44  * 

F-111 

19 

51 

30 

Average 

32 

35 

33 

Bomber 

B-58 

14 

29 

57 

B-70 

23 

24 

53 

Average 

19 

26 

55 

SST 

23 

31 

46  ° 

X-15 

21 

15 

64 

NOTE:  These  data  are  based  on  the  optimal  development 
test  program  except  as  designated. 

*  Based  on  the  actual  development  test  program. 

°  Based  on  total  testing  performed  as  of  end  of  CY  1970. 
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TABLE  31 

TIMING  OF  THE  TEST  PROGRAM 


%  Testing  at  %  Testing  at  %  Testing  at 

Aircraft _ Contract  Award  Design  Freeze _ First  Flight 


Fight  er/Attack 

F-lOO 

6 

81 

100 

F-3H 

21 

64 

100 

F-llA 

20 

47 

53 

F-101 

35 

68 

88 

F-8 

9 

36 

49 

F-105 

29 

100 

100 

F-106 

0 

78 

92 

F-4 

17 

67 

100 

RA-5C 

26 

53 

83 

A- 6  A 

12 

96 

100 

F-111 

21 

53 

83 

A- 7 

19 

42 

45 

Average 

20 

65 

83 

Larger  Aircraft 

B-58 

0 

43 

57 

B-70 

20 

76 

100 

880 

0 

100 

100 

Gulfstream  II 

7 

93 

100 

C-141 

27 

100 

100 

C-5 

42 

100 

100 

Average 

16 

85 

93 

Propeller  Aircraft 

OV-1 

7 

57 

62 

E-2A 

12 

64 

76 

XC-142 

13 

87 

100 

OV-10 

38 

55 

62 

Average 

18 

66 

75 

X-15 

0 

83 

100 

T-38 

12 

60 

80 

AVERAGE  BASED  ON 
ALL  24  AIRCRAFT 


17 


71 


85 
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85%  of  the  testing  recommended  for  the  total  development  program;  only 
15%  of  the  wind  tunnel  test  program  Is  reserved  for  support  of  flight 
testing.  The  regression  equation  should  then  be  increased  by  a  factor 
of  1.  18  to  reflect  the  total  development  facility  test  program. 

One  must  exercise  great  care  in  the  use  of  these  techniques  for 
predicting  optimal  test  programs.  At  best,  they  should  be  considered 
base-line  estimates  to  which  adjustments  can  be  applied.  It  has  already 
been  observed  that  every  aircraft  development  program  is  a  unique  case. 
One  who  uses  these  techniques  should  be  familiar  with  the  particular 
requirements  of  his  program.  Individual  and  company  test  philosophy 
will  continue  to  deviate  from  the  "average";  time  and  dollar  constraints 
will  probably  also  continue  to  affect  test  programs.  Other  types  of  de¬ 
viation  from  the  "average"  wind  tunnel  program  are  suggested  by  the 
B-1  development  program,  where  a  very  long  pre-contract  study  program 
has  shifted  the  test  program  completely  out  of  phase  and  magnitude  with 
these  estimates. 

However,  these  predictors  should  serve  several  useful  purposes. 

For  those  developing  a  new  detailed  test  plan,  they  may  serve  as  a  point 
of  departure.  They  should  provide  a  means  of  measurement  for  evaluation 
of  proposed  test  programs.  The  planners  who  attempt  to  predict  facility 
workloads  for  future  aircraft  should  find  the  techniques  useful.  The 
predictors  presented  here  can  and  should  be  improved.  The  fact  that 
good  correlation  was  obtained  for  so  large  an  assortment  of  aircraft  en¬ 
courages  one  to  believe  that  predictors  within  categories  would  be  even 
better.  Perhaps  predictors  within  given  speed  ranges  and  with  different 
independent  variables  would  show  even  better  correlation.  However,  the 
effort  required  is  beyond  the  limitations  of  this  study. 

The  author  cautions  against  the  use  of  independent  variables  (or 
predictors)  that  do  not  intuitively  suggest  a  relationship  with  the  dependent 
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variable.  A  further  note  of  caution  should  be  exercised  against  the  use 
of  complicated  and  higher  order  mathematical  expressions  which  also 
elude  intuitive  interpretation.  Such  expressions  can  be  forced  to  fit 
the  scatter  in  the  data  to  give  very  good  correlation,  but  may  be  com¬ 
pletely  worthless  as  an  indication  of  trend  or  for  data  outside  the  range 
of  variables.  Even  now,  one  must  be  careful  about  the  use  of  the  re¬ 
gression  equation  outside  the  range  of  the  data  on  which  it  was  based. 
Particularly,  it  appears  that  the  suggested  techniques  may  overpredict 
the  test  needs  of  a  very  large,  high-supersonic  aircraft  with  swing-wings 
and  other  advanced  aerodynamic  features. 

Test  Program  Costs 

As  reported  earlier,  the  data  on  costs  of  testing  were  sketchy  and 
subject  to  various  interpretations.  However,  it  is  desirable  for  planning 
purposes  to  have  some  idea  of  probable  test  costs  on  future  aircraft  de¬ 
velopment  programs. 

The  author  has  completed  the  cost  estimates  for  several  of  the  more 
recent  and  major  military  aircraft  testing  programs  by  using  certain  data 
furnished  in  other  completed  estimates  and  his  own  knowledge  of  wind 
tunnel  test  costs.  Based  upon  eleven  aircraft  development  programs 
and  1970  cost  estimates,  it  appears  that  the  total  test  cost  per  test  hour 
is  between  $1800  and  $3100.  However,  eight  of  the  estimates  were 
between  $2000  and  $2500  per  test  hour.  These  test  costs  include  aero¬ 
dynamic  models,  facility  usage  or  operation  (including  cost  of  govern¬ 
ment  facilities),  data  collection  and  analysis,  and  special  facility 
construction  or  modification  at  the  contractor's  facility,  which  was 
special  for  this  test.  This  last  item  did  not  affect  any  of  the  results. 

Since  the  optimal  test  program  leans  toward  greater  use  of  the  more 
sophisticated  facilities,  it  is  suggested  that  a  figure  of  about  $2500  per 
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tost  hour  be  used  lor  estimating  the  total  cost  of  wind  tunnel  testing 
for  major  aircraft  development  programs.  The  smaller,  low-performance 
aircraft  development  programs  can  probably  use  simple  models  and  more 
inexpensive  facilities,  and  should  cost  perhaps  one-half  the  above 
estimate.  These  estimates  must  change  from  a  1970  cost-base  to  account 
for  future  economic  trends.  "Free"  testing  will  not  change  the  overall 
cost  of  the  test  program,  but  will  be  reflected  in  charges  to  the  develop¬ 
ment  program. 


Summary  of  Test  Program  Predictors 

The  recommended  predicting  procedures  for  an  optimal  wind  tunnel 
test  program  are  summarized  to  make  them  more  accessible  and  to  clarify 
their  usage.  The  following  equation  is  suggested  to  predict  the  total 
wind  tunnel  test  hours  in  a  normal  aircraft  development  program: 

T.  H.  =  24  70  +0,139  T-0.0213W+1.38S 
where: 

T.  H.  =  total  development  test  hours  in  a  wind  tunnel 
T  =  total  maximum  aircraft  sea-level  static  thrust 
(pounds),  with  afterburner,  if  installed: 

W  =  takeoff  gross  weight  (pounds); 

S  =  maximum  speed  at  best  altitude  (miles  per  hour). 
The  predicted  test  program  should  be  increased  by  a  factor  of  about  1.  5 
for  aircraft  which  incorporate  novel  aerodynamic  design  of  a  pioneering 
nature,  aircraft  which  are  expected  to  operate  at  several  extreme  oper¬ 
ating  conditions,  and  aircraft  which  extend  the  performance  envelope 
beyond  standard  technology.  If  the  aircraft  is  of  a  variable-sweep  wing 
design,  the  predicted  test  program  should  be  increased  by  a  factor  of 
two  (after  the  above  correction  is  made). 

The  predicted  test  hours  should  be  phased  in  the  development  pro- 


133 


gram  as  noted: 

15%  prior  to  contract  award, 

70%  prior  to  design  freeze, 

85%  prior  to  first  flight. 

Subdivision  of  the  optimal  wind  tunnel  test  program  by  tunnel  type  (i.  e, , 
subsonic,  transonic,  and  supersonic)  is  possible  with  the  aid  of  Tables 
29  and  30.  Note  that  these  test  hours  do  not  include  development  of 
the  propulsion  system. 

An  estimate  of  testing  costs  can  be  obtained  for  modern  advanced 
aircraft  from  the  predicted  optimal  test  hours  and  a  rate  of  $2500  per 
test  hour.  These  charges  include  wind  tunnel  costs  (including  govern¬ 
ment  facility  charges),  model  costs,  and  contractor  personnel  to  collect 
and  analyze  the  data.  These  estimates  are  based  on  1970  dollars.  Smal¬ 
ler,  low-performance  aircraft  should  cost  only  about  one-half  this  amount. 

The  reader  should  observe  the  restrictions  noted  earlier  in  this 
chapter  before  applying  these  techniques. 
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APPENDIX  A 

LIST  OF  RESPONDENTS  TO  QUESTIONNAIRES 

NAME  RANK  TITLE 

AERONAUTICAL  SYSTEMS  DIVISION 


Beilis,  Benjamin  N. 

Brig.  Gen. 

Boykin,  J.  Arthur,  Jr. 

PL-313 

Daley,  Robert  P. 

Col. 

Esposito,  Alfred  L. 

Brig.  Gen. 

Greenlay,  Robert  R. 

Col. 

Guthrie,  Joseph  A. ,  Jr. 

Col. 

Haviland,  George 

Col. 

Hildebrandt,  James  E. 

Col 

Klepinger,  Richard  H. 

GS-15 

Lengnick,  R.  H. 

Col. 

Martin,  Reese  S. 

Col 

Miller,  William  B. 

GS-16 

Orazio,  Fred  D.  ,  Sr. 

PL-313 

Rail,  F.  T.  ,  Jr. 

GS-15 

Rushworth,  Robert  A. 

Col. 

Sea,  Austin  L. 

GS-16 

Stringer,  Elbert  M. 

Col. 

Tremaine,  Stanley  A. 

GS-15 

Trenholm,  John  B. ,  Jr. 

GS-15 

AERO  PROPULSION  LABORATORY  (A. 

Dunnam,  Marc  P. 

PL-313 

One  other  who  wishes  to  remain  anonymous 

F-15  SPD 

Tech.  Dir. /Weapon  Syst. 

Dep.  for  Eng. 

Dep.  for  Syst.  Mgmt. 

Dir.  Drone  Mgmt. 

Asst.  Subsyst.  Mgmt. ,  0-5  SPO 
Dir.  j  Airframe  Subsyst.  Eng.  Dir. 

A-X  SPD 

Chief,  Aeromech.  Div. ,  Dir.  Airframe 
Subsyst.  Eng. 

F-111  Dep.  SPD 

Dir.  Test  &  Deployment,  Dep.  for  F-111 
Tech.  Dir. ,  Airframe  Subsyst.  Eng. 
Scientific  Dir. ,  Dep.  for  Dev.  Planning 
F-15  Chief  Eng. 

Commander  4950  Test  Wing  (Technical) 
Syst.  Eng.  Dir.  ,  Dep.  for  B-1 
F-5E  SPD 

Supervisory  General  Eng. 

Tech.  Dir.,  B-1  SPO 


Dep.  Dir.  APL 


N 


A 


M 


E 


•RANK 


ARNOLD  ENGINEERING  DEVELOPMENT  CENTER 


Camp,  C.  GS-15 

Croy,  Roy  R.  ,  Jr.  Col. 

Eastman,  Donald  R.  PL-313 

Furlong,  G.  Chester  GS-15 

Glaser,  Leonard  T.  GS-15 

Henry,  Joseph  R.  Col. 

Maynard,  Harry  L.  Col. 

*  Hensel,  R.  W. 


*  Pindzola,  Michael 

*  Potter,  J.  Leith 

*  Schueler,  C.  J. 

*  Whitfield,  Jack  D. 

*  Wimbrow,  William  R. 


FLIGHT  DYNAMICS  LABORATORY 


Antonatos,  Philip  P.  GS-16 

Harney,  Donald  J.  GS-15 

Hoener,  Richard  F.  GS-15 

Lindenbaum,  Bernard  GS-15 


Magrath,  Howard  A.  GS-16 

Scolatti,  Charles  A.  Col. 

Westbrook,  Charles  B.  GS-15 

Zonars,  Demetrius  PL-313 


*  Members  of  ARC,  Inc. ,  Operating  Contractor  at 


T  I  T  L  E 


Chief,  Op.  Analy.  Off.  ,  Dir.  of  Test 

Vice  Commander 

Chief  Scientist 

Tech.  Adv.  to  Dir.  of  Test 

Chief,  Plans,  Programs  &  Policy  Off. 

Dir.  of  Test 

Dir.  of  Technology 

Dir.  of  Operations 

Chief,  Propulsion  Wind  Tunnel  (PWT) 
Chief,  Aerospace  Div. ,  VKF 
Chief,  Aerodynamics  Div. ,  VKF 
Chief,  von  Karman  Gas  Dynamics  Fac. 
(VKF) 

Mgr. ,  Pilot  Tunnels  Br. ,  PWT 


Chief  Flight  Mech.  Div. 

Asst,  for  Experimentation 
Asst,  for  Res.  &  Tech.  ,  Struc.  Diy. 
Dep.  for  Stud.  &  Analy. ,  V/STOL  Tech. 
Div. 

Dir. ,  Vehicle  Dynamics  Div. 

Chief,  Flight  Control  Div. 

Chief,  Control  Criteria  Br. 

Chief  Scientist 
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N 


M 


RANK 


HEADQUARTERS  AIR  FORCE  SYSTEMS  COMMAND 


TIE 


PeterSj  John  D. 

Roe,  Frank  L. ,  Jr. 
Slay,  Alton  D. 

Wimer,  Arthur  G.  ,  Jr. 

HEADQ  UART  ERS 


Brig.  Gen. 
GS-16 
Brig.  Gen. 
PL-313 

U.  S.  AIR  FORGE 


Goldsworthy,  Harry  E.  Lt.  Gen. 

Hargis,  Galvin  B.  ,  Jr.  PL-313 

Kucheman,  H.  B. ,  Jr.  Maj.  Gen. 

NASA 

Baals,  Donald  D.  GS-16 

Becker,  John  V. 

Boatwright,  William  B.  GS-15 

Boswinkle,  Robert  W.  ,  Jr.  GS-16 

Howell,  R.  GS-15 

Jones,  J.  Lloyd 

Kelly,  MarkW.  GS-15 

Lundin,  Bruce  T. 


T  HE  BOE  ING  G  O  M  P  A  N  Y 


Gray,  William  L. 
Miller,  Douglas  S. 
Ruffner,  B.  F. 
Wimpress,  J.  K. 


Dir.  of  Givil  Eng. 

Tech.  Dir.  of  Test.  DGS/Operations 
DCS/  Operations 
Ghief  Scientist 


DGS/Systems  and  Logistics 
Dep.  for  Development,  SAF  R  &  D 
Asst.  DGS/R  &  D 


Asst.  Gh.  ,  High-Speed  Aircraft  Div.  (LRG) 
Gh. ,  Hypersonic  Vehicles  Div.  (LRG) 
Head,  Adv.  Facilities  Res.  Sect.  (LRG) 
Tech.  Asst,  to  Dir.  for  Aero  (LRG) 

Head,  High  Temp.  Str.  Br.  (LRG) 

Research  Asst,  to  Director  (ARG) 

Ghief,  Lg.  Scale  Aero.  Br.  (ARG) 

Director,  Lewis  Research  Genter 


Staff  Eng.  ,  Off.  V.  P.  for  R  &  D 
Aero.  Gonfiguration  Unit  Gh. 

Dir. ,  Product  Res. 

Mgr.,  Airplane  Tech. ,  Aerospace  Gp. 
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A 


M 


E 


TITLE 


N 


_ A  M  E  RANK 

FAIRCHILD  HILLER  CORPORATION 


Kremowski,  John  E. 

Matarazzo,  Aniello 
Neuburger,  H.  Roy 
Rosenthal,  Gordon 
Rubin,  Arnold 
Torrillo,  Dominick  T. 

Williamson,  John  M. 

GENERAL  DYNAMICS  CORPORATION 

Craig,  R.  E. 

Heinemann,  E.  H. 

MacCarthy,  W,  T. 

Madsen,  A.  P. 

Maske,  E.  B. 

Piszkin,  Stanley  T. 

Wild,  John  M. 

GRUMMAN  AEROSPACE  CORPORATION 


Byars,  L.  T. 

Cuffe,  Alfred  G. 
Curtis,  Edward  J. 
Murphy,  William  R. 
Youth,  Stanley 


A-X  Stab.  &  Cont,  Gp.  Leader 
Sr.  Syst.  Eng.  —  Aero. 

Aero.  Staff  Eng. 

Chief  of  Aerodynamics 

A-X  Wind  Tunnel  Test  Coordinator 

Principal  Aero  Systems  Eng. 

Chief  Eng. 


Design  Specialist 

Vice  President 

Eng.  Mgr. ,  Wind  Tunnels 

Chief,  Aerospace  Model  Testing 

Program  Director,  TIP 

Design  Specialist 

Dir.  of  Eng.  Technologies 


Project  Aerodynamicist 

Project  Aerodynamicist 

F-14  Drag  &  Performance  Unit  Ldr. 

Chief  of  Aerodynamics 

Head  of  Aero.  Configuration  Gp. 


name  rank 

LOCKHEED  AIRCRAFT  CORPORATION 


Cleveland,  Frank  A. 

Smelt,  Ronald 
Wilson,  F.  M. ,  Jr. 

McDonnell  aircraft  company 


Baldwin,  Robert  J, 
Barkey,  Herman  D. 
Flesh,  Edward  M. 
Miller,  C.  W. 
Mongold,  Clarence  H. 
Parke,  Darrel  B. 


,T  I  T  L  E 


V.  P.  -  Engineering 

V.  P.  &  Chief  Scientist 

Proj.  Fit.  Sciences  Div.  Mgr. 


V.  P.  ,  Avionics  Engineering 
V.  P. ,  Aircraft  Eng. 
Director,  Design  Eng. 

Mgr.  of  Aerodynamics 
Project.  Aerodynamics  Eng. 
Engineering  Mgr. 


NORTH  AMERICAN  ROCKWELL  CORPORATION 


Blair,  Morgan  M. 

Johnston,  E.  W. 

Ley,  A.  C. 

MacKay,  J.  K.  L. 

Mark,  Leon 
Marshall,  P.  L. 

McCarthy,  John  Francis,  Jr. 
Peterson,  B.  G. 

Peticolas,  R.  P. 

Rose,  Leonard  M. 
Schweiger,  M. 

Stone,  G.  M.  ,  Jr. 

Swanson,  W.  E. 

Wilier,  Jack  E. 


Research  &  Eng.  Staff 
Director,  Flight  Technology 
Aerodynamics  Supervisor 
Technical  Staff 
Project  Eng. 

Mgr.  ,  Aircraft  Development 
V.  P. ,  Systems  Eng. 

Mgr.  ,  Tech.  Support  &  Test  Integration 
Program  Mgr. ,  Trainer  Aircraft 
Mgr.  ,  Technical  Eng. 

Mgr.  ,  R  &  D 

Mgr.  ,  Wind  Tunnels 

V.  P.  ,  Eng.  Aerospace  Gp. 

Eng.  Project  Mgr. 
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NAME  RANK 

NORTHROP  CORPORATION 

Brattj  Robert  W. 

Brown,  S.  H. 

Grogan,  George  C. ,  Jr. 

Howe,  E.  Dabney 
Linder,  D.  W. 

Weyl,  C.  J. 

VOUGHT  AERONAUTICS  DIV.,  LTV 

Clark,  John  Russell 
Evans,  T.  M. 

Isely,  F.  D. 

Louthan,  John  D. 

Prilllman,  F.  W. 

Stahl,  Harold  F. 

Upton,  George  T. 

USAF  SCIENTIFIC  ADVISORY  BOARD. 


TITLE 


Mgr.  ,  Advanced  Design 
Sr.  Technical  Specialist 
V.  P.  ,  Product  Development 
Mgr.  ,  R  &  D 

Mgr.  ,  Fit.  Simulator  Test  Dept. 
Mgr.  ,  Aerodynamics 


Sr.  V.  P.  ,  Technical 
Sr.  Engineer 

Sr.  Engineering  Scientist 
Fit.  Technologies  Mgr. 

Aero.  Project  Eng. 

Aero.  Project  Eng. 

Dir. ,  Engineering  Technologies 

AEROSPACE  VEHICLES  PANEL 


Bogdenoff,  Seymour  M.  Prof.  &  Head,  Gas  Dynamics  Lab, 

Dept.  Aerospace  &  Mech.  Sciences, 
Princeton  University 

Loewy,  Robert  G.  Dean,  College  of  Eng.  &  Applied  Sc. , 

University  of  Rochester 
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APPENDIX  B 

QUESTIONNAIRE— COMPLETE  SHORT  FORM 
FOR  GOVERNMENT  AND  INDUSTRY 


APPENDIX  B 


2  February  1971 


To  the  Respondent: 

This  questionnaire  represents  a  part  of  the  attempt  by  the 
Air  Force  Systems  Command  to  collect  data  which  will  permit 
an  improvement  in  the  aircraft  development  cycle.  The 
primary  thrust  of  this  questionnaire  is  directed  toward  an 
improvement  in  the  usefulness  and  effectiveness  of  the  wind 
tunnels  that  are  used  in  the  development  program. 

You  who  were  chosen  to  answer  this  questionnaire  are  experts 
in  either  systems  development,  flight  testing,  or  facility 
testing.  Many  of  you  have  experience  in  all  three  categories. 

All  of  you  are  recognized  leaders  in  your  profession,  and 
your  opinions  and  judgements  will  combine  to  provide  valuable 
data  and  insight  as  problems  are  investigated  and  solutions 
pondered. 

The  questionnaire  does  not  ask  for  compliments,  complaints, 
or  parochial  views;  instead,  it  seeks  the  truth  as  you  see  it. 

It  is  recognized  that  the  testing  program  is  only  one  of  several 
variables  that  influence  the  success  or  failure  of  a  particular 
aircraft  development.  Only  you  who  have  served  in  positions 
where  you  could  weigh  the  influence  of  the  many  variables  have 
the  exclusive  knowledge  to  contribute  to  this  investigation. 

Your  anonymity  will  be  preserved  to  the  extent  you  desire.  Unles 
you  object,  I  plan  to  list  all  respondees  by  name  and  position 
as  expert  contributors  to  the  study.  I  will  not  attribute 
any  answer  to  you  as  an  individual.  I  would  like  your  name 
on  the  survey  sheet  for  my  own  use  in  case  I  need  to  contact  you 
for  further  clarification  of  some  answers. 

I  realize  that  I  am  asking  for  some  very  difficult  data.  You 
may  sometimes  find  it  impossible  to  provide  simple  answers  as 
requested.  Please  feel  free  to  expand  your  answers  on  the  back 
of  the  sheet  or  use  additional  sheets  as  needed.  I  thank  you 
for  your  cooperation  and  effort. 
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Date: 


Personal  Data 

Name; 

Title: 

Grade  or  Rank: 

Business  Address; 

Business  Telephone: 

Years  Experience  in: 

Aircraft  Development  Programs:  _ yrs. 

Wind  Tunnel  Test  Programs:  yrs. 

Flight  Test  Programs:  yrs. 


Questions  1  through  9  are  directed  toward  an  investigation  of  the  con¬ 
ception  and  development  of  wind  tunnel  test  programs.  Seven  statements 
are  given  in  support  of  a  hypothesis.  It  is  expected  that  there  will  be 
some  exceptions  and  qualifications  to  these  statements,  but  you  should 
indicate  your  general  level  of  agreement  or  disagreement  based  upon  your 
overall  experiences  and  observations. 


1.  Emphasis  on  low  development  cost  and  short  development  time  has 
the  effect  of  encouraging  the  bidder  to  reduce  the  scope  of  his  test 
program  and  thus  increase  risk. 

Disagree  No  Agree 

Strongly _ Disagree _ Opinion _ Agree _ Strongly  _ 

2.  There  is  a  tendency  to  minimize  attention  to  the  development  of  the 
test  plan  because  of  the  many  other  issues  considered  more  important  in 
the  bid  proposal. 

Disagree  No  Agree 

Strongly _ Disagree _ Opinion _ Agree _ Strongly _ 
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3.  The  magnitude  and  quality  of  the  test  plan  in  the  bid  proposal  has 
little  influence  on  the  evaluation  of  the  bid  and  thus  has  little  effect 
upon  selection  of  the  contractor. 

Disagree  No  Agree 

Strongly _ Disagree _ Opinion _ Agree _ Strongly _ . 

4.  The  contractor  does  most  of  the  work  in  preparing  the  test  plan  for 
systems  development. 

Disagree  No  Agree 

Strongly _ Disagree _ Opinion _ Agree _ Strongly _ . 

5.  The  military  developing  agency  exerts  very  little  influence  on  the 
magnitude  or  quality  of  the  test  plan. 

Disagree  No  Agree 

Strongly _ Disagree _ Opinion _ Agree _ Strongly _ . 

6.  Department  of  Defense  test  facility  personnel  exert  very  little  influ¬ 
ence  on  the  magnitude  or  quality  of  the  test  plan. 

Disagree  No  Agree 

Strongly _ Disagree _ Opinion _ Agree _ Strongly _ . 

7.  NASA  test  personnel  exert  very  little  influence  on  the  magnitude  or 
quality  of  the  test  plan. 

Disagree  No  Agree 

Strongly _ Disagree _ Opinion _ Agree _ Strongly _ . 

8.  Do  you  believe  that  some  reorientation  of  emphasis  or  procedure  can 
improve  the  existing  method  of  preparing  and  evaluating  the  test  plan? 

Yes _ No _ . 

9.  If  your  answer  to  (8)  is  "yes,  "  consider  each  of  the  possible  courses 
of  action  noted  below  for  its  potential  to  provide  an  improved  test  plan 
and  test  program  for  systems  development.  Use  the  matrix  to  compare 
each  suggested  change.  For  example,  in  the  first  column  you  will 
compare  alternatives  (a)  and  (b),  in  the  second  column  (a)  and  (c),  etc. 
Place  a  (1)  in  the  appropriate  space  to  indicate  your  perference  of  the 
two  courses  of  action  being  compared;  place  a  (0)  in  the  remaining  space. 
If  you  consider  the  choices  equal,  put  a  (1)  in  both  spaces.  Fill  all 
blanks. 
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a.  Continue  existing  procedures,  but  have  the  military  develop¬ 
ing  agency  attach  less  Importance  to  the  test  plan  in  the  evaluation  of 
the  bid. 

b.  Continue  existing  procedures,  but  have  the  military  develop¬ 
ing  agency  attach  more  importance  to  the  test  plan  in  the  evaluation  of 
the  bid. 


c.  Have  the  company  bid  the  test  plan  only  as  a  lump-sum 
expenditure  and  evolve  the  details  if  and  when  it  gets  the  contract. 

d.  Omit  the  test  plan  from  the  bid  proposal.  Develop  the  test 
plan  after  contract  award  with  the  combined  efforts  of  the  selected  con¬ 
tractor,  Systems  Project  Office,  and  government  test  facility  personnel. 
Add  to  the  contract  on  a  cost-plus-fixed-fee  basis. 


a. 

X 

X 

X 

b. 

X 

X 

X 

c. 

X 

X 

X 

d. 

X 

X 

X 

The  Department  of  Defense  has  directed  the  Military  Services  to  make  some 
adjustments  in  the  management  of  weapon  systems  acquisition.  A  part  of 
this  philosophy  concerns  a  thorough  flight  evaluation  of  the  aerospace 
system  and  its  relation  to  tooling  and  production  go-ahead.  However, 
in  this  part  of  the  survey  the  questions  are  directed  toward  the  contribu¬ 
tions  to  be  made  by  the  test  facility  prior  to  completion  of  the  flight 
vehicle.  The  general  thrust  of  the  DoD  directive  is  to  invest  development 
dollars  to  save  production  dollars;  i.  e.  ,  to  find  and  correct  mistakes  and 
make  appropriate  trade-offs  early  in  the  development  cycle  where  the  pen¬ 
alty  is  not  SO  high.  Emphasis  is  on  demonstration  of  performance  and  re¬ 
duction  of  technical  risk  before  commitment  of  funds  for  full-scale  develop¬ 
ment.  Questions  10  through  14  concern  the  role  of  the  test  facility  as  it 
relates  to  this  policy. 

10.  Do  you  agree  with  the  new  policy  and  emphasis? 

Strongly  No  Agree 

Disagree _ Disagree  _ Opinion  _ Agree  _ Strongly _ . 


Comment: 


11.  It  is  hypothesized  that  this  shift  in  emphasis  in  the  development 
cycle  will  permit  the  test  facility  to  be  more  useful  and  influence  the 
system  development  more  strongly.  Do  you  agree? 

Strongly  No  Agree 

Disagree _ Disagree _ Opinion _ Agree _ Strongly  _ 

Comment: 


12.  The  shift  in  emphasis  in  development  will  require: 

_  more  facility  testing. 

_ _  less  facility  testing. 

_  no  change  in  facility  test  load. 

13.  Two  important  decision  points  arise  in  the  development  cycle  prior 
to  full-scale  development.  One  decision  point  is  at  the  end  of  the 
conceptual  phase  and  one  at  the  end  of  the  validation  phase.  The  de¬ 
cision  to  be  made  at  each  point  is  whether  or  not  the  program  is  ready 
to  progress  into  the  next  phase.  The  two  phases  are  briefly  described 
below.  In  keeping  with  the  philosophy  of  illustrating  progress  with 
hardware  and  performance  demonstrations,  please  comment  on  the  pos¬ 
sible  contribution  of  the  test  facility  at  these  decision  points.  In 
other  words,  if  you  were  the  decision  maker,  what  information  would 
you  like  that  could  be  derived  from  the  test  facility?  Be  as  specific  as 
possible  and  note  any  changes  in  timing  you  may  prefer  regarding  when 
certain  data  are  taken  in  the  development  cycle. 

Conceptual  Phase.  This  is  the  first  phase  in  the  development 
cycle  and  includes  identification,  definition  and  analysis  of  conceptual 
systems.  Also  included  is  experimentation  and  test  of  operational  re¬ 
quirements,  key  components,  critical  subsystems  and  marginal  technology. 
Major  uncertainties  are  identified  and  a  development  program  and  preferred 
system  are  proposed. 
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Comment: 


Validation  Phase.  This  is  the  second  phase  and  emphasizes 
hardware  development  and  evaluation  to  resolve  or  minimize  technical 
risks.  The  costs,  schedules,  performance  and  military  needs  defined 
in  the  conceptual  phase  are  evaluated  by  the  contractors  who  will  do 
the  full-scale  development.  The  decision  at  the  end  of  this  phase  may 
be  either  to  cancel  the  development,  proceed  into  full-scale  development, 
or  perhaps  defer  full-scale  development  until  certain  critical  high-risk 
components  are  developed  or  technical  problems  resolved. 

Comment: 


14.  It  is  considered  poor  management  practice  to  place  individuals  or 
groups  in  a  position  where  they  may  be  required  to  make  decisions  against 
their  own  best  interests.  Because  the  possibility  of  program  cancellation 
is  an  alternative  at  each  decision  point,  some  parties  have  more  to  lose 
than  others  and  thus  have  an  incentive  to  interpret  the  experimental  data 
in  such  a  way  that  the  program  will  look  good.  It  is  not  the  intent  of 
these  statements  to  question  anyone's  honesty  or  integrity,  but  instead 
to  orient  the  procedure  in  keeping  with  good  management  principles. 

Please  comment  on  what  you  think  would  be  a  desirable  procedure  for 
analyzing  and  reporting  on  experimental  data  from  the  wind  tunnel  that 
is  to  be  used  in  ascertaining  compliance  with  expected  performance. 
Include  the  roles  of  at  least  three  parties  in  your  comments;  i.  e.  ,  the 
contractor.  Systems  Project  Office,  and  DoD  test  facility  personnel. 

Comment: 


Most  of  the  large  aeronautical  test  facilities  in  the  United  States 
were  built  in  the  early  1950's.  It  is  fact  that  these  facilities  often  do 
not  provide  the  test  environment  desired  for  development  of  advanced 
weapon  systems.  Questions  15  through  20  deal  with  this  situation,  its 
effects,  and  possible  remedies. 
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15.  Has  the  deficiency  in  environmental  simulation  in  aeronautical 
test  facilities  had  detrimental  effects  upon  systems  development? 

None _ ,  Some _ Much _ ,  Very  Much _ . 

Briefly  comment  on  any  specific  examples  you  may  know. 


16.  If  test  facility  capability  is  not  improved,  what  will  be  the  effect 
on  aeronautical  systems  development  over  the  next  decade?  Perform¬ 
ance  limitations?  More  costly  development  programs?  Briefly  explain 
any  consequences  you  may  see. 


17.  There  is  a  belief  among  some  government  and  industry  personnel  that 
the  government  should  provide  the  high  cost-low  use  aeronautical  test 
facilities  for  everyone's  use,  while  industry  should  have  its  own  back¬ 
yard  low  cost-high  use  facilities.  Do  you  agree  with  this  general  policy? 

Strongly  No  Agree 

Disagree _ Disagree _ Opinion _ Agree _ Strongly _ , 

Comment: 


18.  If  you  agree  with  the  policy  stated  in  Question  17,  what  do  you  think 
is  a  rule-of-thumb  dividing  line  between  the  two  types  of  facilities?  Use 
dollars  based  upon  today's  construction  costs.  $ _ . 

If  you  think  that  some  other  measure  than  cost  should  be  the  divid¬ 
ing  line,  give  your  rationale. 

If  you  do  not  agree  with  the  general  policy,  please  comment  and 
give  an  alternative  if  you  have  one. 
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19.  Do  you  believe  that  company-owned  aeronautical  test  facilities 
help  the  company  in  obtaining  government  contracts  ? 

None _ ,  Some _ ,  Much _ Very  Much _ . 

Does  this  belief  hold  true  even  though  the  same  capability  is  available 
for  company  use  at  a  government  test  facility?  Yes _ ,  No _ . 

Comment: 


20.  You  are  probably  aware  that  it  may  take  5  to  7  years  to  design,  con¬ 
struct  and  bring  to  operational  usefulness  a  large  aeronautical  test 
facility.  Such  a  facility  cannot  contribute  to  the  development  of  a  weapon 
system  if  it  is  brought  along  in  a  parallel  development  program.  Yet  within 
the  DoD,  the  present  justification  and  funding  system  requires  that  the 
facility  be  closely  tied  to  the  test  needs  of  an  aircraft  development  pro¬ 
gram  that  carries  a  rather  firm  commitment.  Please  make  any  comments 
you  have  regarding  this  dilemma.  How  should  procedures  be  changed? 
What  do  you  believe  is  the  soundest  way  to  plan  and  fund  large  aeronau¬ 
tical  test  facilities  to  best  meet  test  requirements  and  with  least  risk  of 
building  facilities  which  are  not  needed  ? 


21.  Do  you  believe  that  the  DoD  test  facilities  can  better  serve  the 
aeronautical  development  programs  by  providing  experienced  personnel 
on-site  to  provide  additional  services;  e.g.  ,  guidance  on  the  test 
program,  data  analysis,  recommendations,  etc.  ?  Yes _ No _ 


Comment: 
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22.  It  is  hypothesized  that  a  very  subtle  change  in  aircraft  design  is 
being  caused  by  the  growing  deficiencies  in  the  test  facilities.  It  is 
suggested  that  aircraft  are  sometimes  designed  to  fit  facility  capability 
at  the  expense  of  optimum  performance.  It  is  suggested  that  compromise 
designs  are  accepted  because  facilities  are  not  available  to  verify  new 
ideas  or  theories  with  any  degree  of  confidence.  Such  compromises  in 
the  aircraft  are  never  attributed  to  facility  deficiencies,  for  a  conflict 
never  arises  between  test  requirements  and  facility  capability. 

Do  you  agree  with  the  hypothesis: 

Strongly  No  Agree 

Disagree _ Disagree _ Opinion _ Agree _ Strongly _ 

Briefly  note  any  examples  from  your  experience  or  knowledge  which  sup¬ 
port  this  hypothesis: 


23.  It  is  hypothesized  that  the  aircraft  development  program  could  be 
improved  and  cost  savings  incurred  if  more  effort  were  devoted  to  the 
wind  tunnel  test  program  earlier  in  the  cycle,  particularly  prior  to  design- 
freeze  on  the  first  flight  vehicle.  Do  you  agree? 

Strongly  No  Agree 

Disagree _ Disagree _ Opinion _ Agree _ Strongly _ 

Comment: 


24.  Many  aircraft  development  programs  experience  further  wind  tunnel 
testing  during  the  flight  evaluation  phase  to  investigate  and  correct 
deficiencies  discovered  during  flight.  Corrections  and  changes  are  more 
costly  at  this  point  than  those  made  earlier  in  the  cycle.  Some  of  these 
deficiencies  might  have  been  prevented  with  a  more  thorough  facility 
test  program,  but  with  attendant  increases  in  cost  and  time  of  that  phase 
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of  the  program.  Briefly  discuss — in  a  qualitative  manner  if  necessary — 
your  findings  on  the  relative  costs  of  more  facility  testing  versus  undis¬ 
covered  design  deficiencies,  and  express  any  ideas  you  may  have  to 
optimize  this  trade-off. 


25.  What  have  you  found  to  be  the  major  constraints  that  influence  the 
quantity  and  quality  of  the  wind  tunnel  test  program  in  aircraft  develop¬ 
ment  (e.  g. ,  time,  money,  etc.)?  If  more  than  one,  list  in  order  of 
decreasing  importance.  Comment  on  specific  development  programs  if 
you  have  such  data. 


26.  When  none  of  the  test  facilities  can  provide  the  desired  simulation 
parameters,  similar  tests  are  sometimes  performed  in  different  facilities 
in  an  attempt  to  verify  certain  data.  In  your  opinion,  what  part  of  the 

modern  test  program  is  motivated  by  such  facility  deficiencies  ?  _ 

percent.  Examples? 


27.  The  functions  which  denote  categories  of  use  of  the  wind  tunnel  are 
as  follows: 

a.  Verify  the  design  hypotheses;  i.  e. ,  assure  that  theory  and 
past  experiences  have  been  applied  correctly  and  produce  expected 
results. 

b.  Generate  new  design  information  and  improvements  in  the 
design  concept;  i.  e.  ,  produce  new  technology  to  be  applied  to  the 
system. 


c.  Expose  difficulties  which  may  have  been  overlooked;  i.  e.  , 
look  for  undefined  and  unexpected  problems. 
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How  do  you  rank  the  relative  Importance  of  these  functions  in  the  aircraft 
development  program? 

Most  Important:  _ 

Second:  _ 

Third:  _ 

If  you  do  not  think  that  these  three  functions  adequately  cover  reasons 
for  testing,  please  add  others  and  rank. 


Would  you  like  a  summary  of  the  results  of  this  survey?  Yes _ No 
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QUESTIONNAIRE— LONG  FORM  FOR  INDUSTRY 
(First  25  Questions) 
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1  February  1971 


To  the  Respondent: 

This  questionnaire  represents  a  part  of  the  attempt  by  the  Air  Force  Systems 
Command  to  collect  data  which  will  permit  an  improvement  in  the  aircraft 
development  cycle.  The  primary  thrust  of  this  questionnaire  is  directed 
toward  an  improvement  in  the  usefulness  and  effectiveness  of  the  wind  tun¬ 
nels  that  are  used  in  the  development  program.  Some  of  the  data  requested 
can  be  obtained  from  historical  documentation.  Other  data  are  not  so  ob¬ 
jective,  and  are  available  only  in  your  mind  as  you  apply  your  experience 
and  value  system.  The  first  25  questions  are  directed  toward  an  evalua¬ 
tion  of  a  particular  aircraft  development  program.  The  remainder  should 
be  answered  on  the  basis  of  your  total  experience.  The  answers  to  the 
first  12  questions  are  essentially  historical,  and  it  is  expected  that  all 
those  who  are  evaluating  the  same  aircraft  will  collaborate  in  obtaining 
this  data.  However,  you  are  asked  to  provide  independent  answers  to  all 
those  questions  after  number  12.  I  will  discuss  these  answers  with  you 
and  any  others  who  may  answer  the  questionnaire  for  this  particular  air¬ 
craft  in  a  joint  meeting  at  a  later  date.  At  that  time  we  will  try  to  resolve 
or  explain  any  major  differences  of  opinion. 

The  questionnaire  does  not  ask  for  compliments,  complaints,  or  parochial 
views;  instead,  it  seeks  the  truth  as  you  see  it.  It  is  recognized  that 
the  testing  program  is  only  one  of  several  variables  that  influence  the 
success  or  failure  of  a  particular  aircraft  development,  and  only  you  who 
were  on  the  scene  can  distinguish  among  these  influences.  You  are  urged 
to  give  thoughtful  consideration  to  your  answers  for  you  hold  exclusive 
knowledge  through  which  performance  of  the  testing  program  can  be 
measured. 

Your  anonymity  will  be  preserved  to  the  extent  you  desire.  Unless  you 
object,  I  plan  to  list  all  the  respondees  by  name,  title,  and  company  as 
expert  contributors  to  the  study.  I  will  not  attribute  any  answer  to  you  as 
an  individual.  I  would  like  your  name  on  the  survey  sheet  for  my  own  use 
in  case  I  need  to  contact  you  for  further  clarification  of  some  answers. 

I  realize  that  I  am  asking  for  some  very  difficult  data.  You  may  sometimes 
find  it  imfiossible  to  provide  simple  answers  as  requested.  Please  feel 
free  to  expand  your  answers  on  the  back  of  the  sheet  or  use  additional 
sheets  as  needed.  I  thank  you  for  your  cooperation  and  effort. 


154 


JAMES  G.  MITCHELL 
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Date: 


Personal  Data 


Name: 

Company: 

Job  Title  or  Position: 

Business  Address: 

Business  Telephone: 

Years  experience  in: 

Aircraft  Development  Programs:  _ yrs. 

Wind  Tunnel  Test  Programs:  _ yrs. 

Flight  Test  Programs:  _ yrs. 


Aircraft  Development  Program 

1.  Name  of  aircraft  being  evaluated: 

2.  Your  position  on  this  program: 

3.  Program  began  (contract  signed):  month^_^ _ ,  year _ . 

Aircraft  first  flight:  month _ ,  year _ . 

Production  go-ahead:  month _ ,  year _ . 

Program  conclusion:  month _ ,  year _ . 

The  following  questions  are  directed  toward  a  breakdown  of  the  wind 
tunnel  test  program  into  selected  categories.  You  should  include  only 
the  testing  associated  with  the  development  program;  i.  e. ,  exclude  any 
retrofit  or  follow-on  test  programs  which  may  have  been  performed  after 
development  unless  these  data  are  specifically  requested.  The  "wind 
tunnel  test  program"  is  to  include  all  aerodynamic  or  structural  testing 
in  wind  tunnels  which  excludes  development  of  the  actual  propulsion 
system.  Propulsion  -  aircraft  integration  type  tests  are  to  be  included. 
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4.  At  what  date  did  the  wind  tunnel  program  make  its  last  contribution 

to  the  first  flight  vehicle;  i.  e. ,  when  was  design  frozen  on  the  first 
flight  vehicle  ?  month _ _ ,  year _ . 

5.  Wind  tunnels  are  usually  classified  as  subsonic,  transonic,  and 
supersonic  with  some  overlap  in  capability.  Please  designate  the  total 
number  of  test  hours  in  support  of  the  development  program  in  each  type 
of  facility  and  indicate  the  owner  (or  location)  of  the  facility. 

Subsonic  Transonic  Supersonic 


DOD  _  _  _ 

NASA  _  _  _ 

Own  Facilities  _  _  _ 

Other  Industry  Facilities  _  _  _ 

Other  (please  specify)  _  _  _ 

TOTALS:  _  _  _ 

6.  How  many  wind  tunnels  did  you  use  on  this  program? 

Subsonic _ ,  Transonic _ ,  Supersonic _ . 

7.  Designate  the  cumulative  wind  tunnel  test  hours  that  may  be  attribu¬ 
ted  to  this  program  within  the  following  restrictions: 

Before  contract  award _ 

At  design-freeze  on  the 
first  flight  aircraft  (ref. 

Question  No.  4)  _ 

At  the  event  of  first 

flight  _ 


8.  How  many  additional  test  hours  have  been  collected  in  support  of 
retrofit  and  follow-on  development  programs? 

Subsonic _ ,  Transonic _ ,  Supersonic _ 
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THE  NEXT  FOUR  QUESTIONS  ARE  CONCERNED  WITH  WIND  TUNNEL  COSTS: 

9.  Wind  tunnel  costs  are  usually  figured  on  an  hourly  basis  and  are 
primarily  a  function  of  the  wind  tunnel  size  and  performance.  Please 
note  the  amount  of  development  testing  accomplished  in  each  category 
of  price  ranges.  Base  your  estimates  on  today's  costs.  If  you  are  not 
aware  of  the  present  facility  charge  rate,  use  the  index  at  the  end  of 
this  questionnaire  to  calculate  a  charge  rate  for  Calendar  Year  1971. 


Under  $800/hr: 

hrs. 

$801  -  $  1500/hr: 

hrs. 

$1501  -  $  3000/hr: 

hrs. 

Over  $  3000/hr: 

hrs. 

10.  What  was  the  cost  of  the  wind  tunnel  test  program  in  support  of  the 

system  development?  $ _ . 

11.  How  were  these  costs  broken  down? 

Aerodynamic  models  $ _ . 

Facility  usage  or 

operation  $ _ . 

Data  collection  and 

analysis  $ _ . 

Facility  construction 
or  modification  which 
was  special  for  this 

development  $ _ . 

Other  (please 

specify)  $ _ . 

12.  Were  there  other  wind  tunnel  test  costs  that  you  did  not  pay  and 

cannot  estimate;  e.  g.  ,  operation  of  government  facilities?  Yes _ 

No _ .  Explain: 
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13.  Compare  this  development  program  with  others  you  have  experienced. 
For  those  technical  areas  amenable  to  investigation  in  the  wind  tunnel, 
assess  the  level  of  technical  difficulty  (advancement  of  the  state-of-the- 
art)  at  the  time  of  development. 

High _ ,  Medium _ ,  Low _ . 


Comment: 


14.  Were  there  constraints  that  influenced  the  quantity  and  quality  of 
this  wind  tunnel  test  program  (e.  g.  ,  time,  money,  outside  influence, 
etc.)?  If  more  than  one,  list  in  order  of  decreasing  importance.  Explain: 


15.  Compare  this  development  program  with  others  you  have  known  and 
rate  its  success.  Was  the  development  program  successful  in  meeting 
technical  objectives  in  a  timely  and  economical  manner?  Ignore  the 
ultimate  usefulness  or  success  of  the  system  itself  and  evaluate  only  the 
development  program. 

Poor _ ,  Fair _ ,  Average _ ,  Good _ ,  Very  Good _ . 

16.  In  your  opinion,  how  strongly  did  the  wind  tunnel  test  program  con¬ 
tribute  to  the  success  or  failure  of  the  development  program?  Explain, 
and  note  other  major  contributing  influences  on  this  particular  development 
program. 

Very  little _ ,  Moderately _ ,  Very  much _ . 

17.  Did  you  experience  difficulty  in  obtaining  certain  data  on  this 
program  because  of  test  facility  deficiencies  ?  Were  certain  simulation 
parameters  inadequate?  Explain: 
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18.  When  none  of  the  facilities  can  provide  the  desired  simulation  para¬ 
meters,  similar  tests  are  sometimes  performed  in  different  facilities  in 
an  attempt  to  verify  certain  data.  Do  you  think  some  of  your  testing  was 

motivated  by  such  a  facility  deficiency?  _ .  If  so,  how  much? 

_ per  cent. 


Please  try  to  place  yourself  in  time  once  again  at  the  beginning  of  this 
development  program,  but  give  yourself  benefit  of  the  knowledge  you  have 
gained  during  and  since  this  program.  You  have  the  opportunity  to  origi¬ 
nate  and  implement  a  wind  tunnel  test  program  of  your  own  choosing.  You 
may  change  the  magnitude  of  the  program  and  shift  emphasis  within  the 
program  to  provide  what  you  believe  to  be  the  optimal.  If  you  are  enlarg¬ 
ing  the  test  program,  use  your  own  judgement  of  the  value  of  additional 
data  as  compared  with  the  additional  time  and  cost  of  testing  .  Do  not  be 
limited  by  any  original  constraints  which  may  have  dictated  the  facility 
test  program.  You  may  need  to  exercise  care  in  evaluating  the  usefulness 
of  the  test  data  where  you  were  searching  for  verification  of  your  design 
hypotheses.  It  would  be  easy  in  retrospect  to  eliminate  any  test  program 
which  only  confirmed  what  you  already  thought.  Please  try  to  recall  the 
uncertainty  and  risk  which  prevailed  at  that  time.  Questions  19  through 
25  are  concerned  with  your  definition  of  the  optimum  facility  test  program 
for  this  particular  system  development. 

19.  In  your  opinion,  the  optimal  wind  tunnel  test  program  would  have 
been  about  test  hours  and  distributed  as  follows: 


Subsonic 

hrs. 

Transonic 

hrs. 

Supersonic 

hrs. 

20.  The  optimal  wind  tunnel  test  program  would  provide  most  usefulness 
with  the  following  distribution: 

Before  contract  award _ hrs. 

At  design -freeze  on 
the  first  flight  air¬ 
craft  (ref.  Question 

No.  4)  _ hrs. 


hrs. 


At  the  event  of  first 
flight 
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21.  The  optimal  wind  tunnel  test  program  would  be  distributed  among 
facilities  as  follows: 


Under  $800/hr: 

hrs. 

$801  -  $  1500/hr: 

hrs. 

$1501  -  $3000/hr: 

hrs. 

Over  $3000/hr: 

hrs. 

22.  What  would  be  the  approximate  cost  of  your  new  optimal  wind  tunnel 

test  program?  $ _ . 

23.  Do  you  believe  that  implementation  of  your  hypothetical  test  pro¬ 
gram  would  have  resulted  in: 


Less  overall  development  cost? 

Yes 

_  No_ 

Superior  system  performance? 

Yes _ 

_  No  _ 

Shorter  time  to  system  demon¬ 

stration  ? 

Yes 

No  _ 

Less  flight  testing  required? 

Yes 

_  No  _ 

24.  Were  there  deficiencies  revealed  in  the  flight  evaluation  of  this 
aircraft  that  your  optimal  test  program  might  have  minimized  or  prevented 
(performance,  control,  structural,  etc.)?  _ .  If  so,  explain. 


25.  The  functions  which  denote  categories  of  use  of  the  wind  tunnel  are 
as  follows: 

a.  Verify  the  design  hypotheses;  i.  e.  ,  assure  that  theory  and 
past  experiences  have  been  applied  correctly  and  produce  expected 
results. 


b.  Generate  new  design  Information  and  improvements  in  the 
design  concept;  i.  e. ,  produce  new  technology  to  be  applied  to  the  system. 
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c.  Expose  difficulties  which  may  have  been  overlooked;  i.  e. , 
look  for  undefined  and  unexpected  problems. 

In  the  aircraft  development  program  being  evaluated,  what  was  the  rela¬ 
tive  importance  of  the  three  categories  ?  In  your  optimal  program,  what 
do  you  think  should  be  more  important?  Rank  the  functions  in  decreasing 
order  of  importance. 

Past  Program  Optimal  Program 


a. 


b. 


c. 


If  you  do  not  think  that  these  three  functions  adequately  cover  your 
reasons  for  testing,  please  add  others  and  rank. 


APPENDIX  D 


EXHIBITS  OF  COMMENTS  BY  RESPONDENTS 


EXHIBIT  1 


COMMENTS  FROM  THE  QUESTIONNAIRES  ON 
TEST  FACILITY  SUPPORT  DURING  THE 
CONCEPTUAL  PHASE  OF  DEVELOPMENT 


"Define  the  aerodynamic  characteristics  essential  to  predicting  design 
mission  performance,  including  take-off  and  landing  high-lift  preliminary 
performance.  Provide  inlet  testing  sufficient  to  establish  the  feasibility 
of  attaining  the  required  installed  performance  levels.  Perform  airloads 
tests  of  sufficient  scope  to  enable  making  preliminary  estimates  of  primary 
structure  critical  loads.  " 

"Evaluate  the  contractor's  technical  concepts,  approach  to  problem  iden¬ 
tification  and  solution,  and  probable  success  in  the  wind  tunnel.  In  the 
flight  simulator,  evaluate  the  contractor's  flight  control  configuration  con¬ 
cept  and  ability  to  make  trade-offs.  Evaluate  engine  performance  in  a 
propulsion  test  call.  " 

"The  primary  contribution  of  the  test  facility  at  this  point  is  to  furnish 
the  data  which  allows  the  determination  of  what  is  possible  to  achieve 
in  various  technical  areas  of  interest  to  the  particular  system.  Examples 
would  be  materials  data,  aerodynamic  efficiency,  engine  performance, 
sensor  and  display  resolution--all  of  which  might  be  expected  to  be  avail¬ 
able  so  that  some  indication  of  the  expected  system  performance  and 
probability  of  achieving  this  performance  can  be  made.  In  general,  the 
more  data  the  higher  the  probability  of  success.  " 

"The  role  of  the  wind  tunnel  must  be  in  providing  accurate  parametric 
data  on  pertinent  configurations  to  assure  that  the  data  used  for  compara¬ 
tive  analyses  are  well  founded.  " 

"More  analysis  and  preliminary  design  is  accomplished  at  this  time  than 
any  other.  For  a  new  configuration,  introduction  of  new  technologies  and 
realistic  estimates  of  the  performance  of  a  proposed  system  in  a  test 
facility  is  definitely  required  to  obtain  aerodynamic  flow  phenomena, 
aerodynamic  loads,  and  to  investigate  sensitive  areas.  The  data  obtained 
can  be  used  for  trade-off  studies,  identifying  uncertainties  and  providing 
numbers  for  describing  sensitivity  areas.  " 

"Even  this  phase  should  be  backed  up  by  some  hard  test  data.  All  too 
often  the  Air  Force  specifies  a  matrix  of  aircraft  with  various  trades  and 
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EXHIBIT  1— Continued 


options.  As  a  result,  there  usually  is  not  a  solid  design  in  the  bunch.  " 

"Small-scale  model  tests  of  the  airframe  are  needed  to  indicate  aero¬ 
dynamic  parameters  and  other  performance  data  expected.  Stability 
data  is  needed  to  identify  any  problem  areas.  Control  data  could  be 
needed  in  certain  cases.  All  data  should  cover  the  Mach  number  range 
expected  in  the  aircraft.  " 
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EXHIBIT  2 

COMMENTS  FROM  THE  QUESTIONNAIRES  ON  TEST 
FACILITY  SUPPORT  DURING  THE  VALIDATION 
PHASE  OF  DEVELOPMENT 


"There  should  be  extensive  testing  over  the  entire  Mach  number/angle- 
of-attack  range  on  models  which  are  heavily  instrumented  to  obtain  data 
for  detailed  flow  diagnostics.  There  should  be  complete  documentation 
of  the  steady-state  and  dynamic  aerodynamic  characteristics  of  the  clean 
airplane  and  high  lift  configuration — including  aeroelastic  effects.  Com¬ 
ponent  loads  and  airloads  distributions  tests  should  be  performed  over  the 
complete  range  of  operating  conditions  to  enable  accurate  definition  of 
design  loads  for  primary  structure  and  a  major  portion  of  the  secondary 
structure.  If  applicable,  perform  store-separation  tests  to  assure  com¬ 
patibility  of  store-airframe  combination.  " 

"In  a  wind  tunnel  evaluate  the  contractor's  aerodynamic  configuration  to 
assure  that  it  is  consistent  with  all  design  requirements  (not  just  per¬ 
formance  requirements).  In  a  flight  simulator  evaluate  flying  qualities 
of  the  contractor's  entry,  considering  failure  cases,  realibility,  etc.  In 
a  propulsion  test  cell  evaluate  engine  performance,  effects  of  bleed,  and 
nacelle  environment  (inlets  and  nozzle).  " 

"We  need  model  tests  at  sufficiently  large  scale  to  tune  flaps  and  con¬ 
trols,  determine  lift/drag  at  trim,  and  obtain  realistic  values  of  perform¬ 
ance,  stability  and  control.  The  test  program  must  include  dynamic  sta¬ 
bility  tests  if  inertial  characteristics  indicate  a  questionable  situation. 
The  same  is  true  for  buffeting  and  aeroelastic  problems.  Engine  perform¬ 
ance  should  be  determined  in  tests  and  sensitivity  to  engine  attitude  and 
flow  distortion  determined.  Airframe-inlet  integration  problems  might 
require  special  tests.  In  high  speed  aircraft,  heat  transfer  tests  in  local¬ 
ized  regions  might  be  required.  " 

"Test  data  on  critical  components  will  help  to  determine  if  the  risks  are 
in  some  way  compatible  with  the  expected  benefits.  Every  possible  risk 
cannot  be  eliminated.  " 

"The  decision-maker  should  be  satisfied  with  large-scale  data  from  test 
facilities.  He  should  expect  marginal  technology  to  be  reasonably  well 
validated  through  accepted  test  techniques.  " 
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EXHIBIT  2— Continued 

"The  validation  must  be  of  a  configuration (s)  which  can  be  practically 
developed  into  a  piece  of  hardware.  " 

"The  role  of  the  wind  tunnel  is  to  provide  comprehensive  and  specific 
data  on  the  selected  configuration  to  assure  soundness  of  design.  Analy¬ 
ses  of  structural  integrity,  performance,  flying  qualities,  etc.  must  be 
based  on  wind  tunnel  tests  of  models  which  simulate  the  proposed  design 
as  closely  as  possible.  " 

"It  must  be  certain  that  at  least  one  base-line  has  been  thoroughly  tested 
and  analyzed.  " 

"All  items  must  work  properly  in  a  validation  test  before  going  on  to  full- 
scale  development.  Since  test  conditions  may  be  more  realistic  than  in 
previous  testing,  some  failure  can  be  expected.  When  these  occur,  test¬ 
ing  should  be  shifted  to  document  problems  and  then  stopped.  When  cor¬ 
rections  are  made.  Validation  Testing  should  continue  or  start  over.  " 

"At  this  point  there  should  be  hard  test  data  on  the  final  design.  All  too 
often,  the  contractor  is  still  'incrementing'  and  doesn't  have  final  results.  " 

"The  answers  from  the  test  facility  can  be  used  to  design  the  later  flight 
tests.  " 
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EXHIBIT  3 

COMMENTS  FROM  THE  QUESTIONNAIRES  ON  MANAGEMENT 
OF  TEST  FACILITY  VALIDATION  DATA 


"The  SPO  and  DoD  test  personnel  must  develop  test  plan  jointly.  Test 
facility  personnel  make  recommendations  to  SPO  as  a  result  of  tests^ 
but  also  send  a  copy  to  SPO's  boss.  " 

"Bring  the  contractor,  SPO  to  best  Federal  facilities.  The  Federal  Govern¬ 
ment  is  the  buyer  and  accordingly  must  accept  responsibility  for  quality 
and  performance,  " 

"Some  day,  aerodynamic  performance  data  certified  by  a  government 
laboratory  will  be  required  as  presently  done  for  engine  performance  data. 
The  knowledge  that  certification  will  be  done  will  aid  an  initial  interpre¬ 
tation.  " 

"When  we  reach  a  point  in  development  where  the  SPO  and  DoD  test  facil¬ 
ity  personnel  work  as  a  team,  the  problem  will  be  taken  care  of.  " 

"Both  the  contractor  and  SPO  are  subject  to  these  pressures.  The  test 
facility's  role  should  be  to  assist  the  other  two  parties  in  solving  prob¬ 
lems.  If  test  facility  personnel  became  the  judge  of  compliance,  this 
role  would  be  compromised  since  the  other  two  parties  would  be  tempted 
to  hide  problems.  The  best  solution  is  an  independent  evaluation  board 
within  DoD.  " 

"The  SPO  should  lead  a  team  of  government  experts  in  accessing  facility 
data.  Government  test  facility  personnel  should  be  instrumental  on  this 
team.  " 

"Experimental  data  should  be  tracked  by  all  participating  parties  on  a 
continuing  basis.  Data  review  meetings  can  'snow'  even  good  and  ex¬ 
perienced  people,  and  usually  only  one  party's  views  are  heard.  " 

"The  SPO  should  have  overall  responsibility,  but  the  key  to  identifying 
and  solving  problems  is  communication  between  the  SPO  and  test  facility 
engineer.  The  procedures  should  not  permit  the  test  engineer  to  turn  his 
data  over  to  the  SPO  and  forget  it.  The  test  engineer  should  participate 
in  formulating  the  test  program,  conduct  and  report  tests,  and  participate 
in  interpreting  tests  with  SPO.  The  contractor,  working  independently  to 
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some  extent,  should  end  up  with  the  same  interpretation.  " 

"The  contractor  would  have  prime  responsibility  for  reduction,  initial 
analysis  and  presentation  of  test  data.  These  results  should  be  collated, 
reviewed  and  analyzed  by  SPO,  with  significant  technical  assist  from 
technical  experts  in  discipline-oriented  laboratories  or  organizations. 

Test  facility  personnel  do  not  play  a  significant  role  I" 

"I  don't  think  any  system  will  insure  that  the  contractor  or  SPO  will  be 
objective  in  considering  project  cancellation,  and  I  wouldn't  waste  time 
trying  to  design  such  a  system.  The  job  of  the  contractor  and  SPO  is  to 
make  the  project  go  and  they  should  not  be  burdened  in  assessing  whether 
to  stop  it.  DoD  test  personnel  can  be  more  objective.  NASA  can  be  even 
more  so  and  should  be  the  source  of  an  independent  assessment  on  how 
the  project  is  going.  DAG  type  committees  can  be  effective,  but  they 
need  new  faces  and  more  time  than  usually  used  at  present.  " 

"Contractor,  SPO  and  test  facility  personnel  are  'involved'  and  are  under 
pressure  to  keep  the  project  going.  We  need  outside  review;  i.  e. ,  sig¬ 
nificant  expertise  from  outside  DoD  organization.  " 

"DoD  test  facility  personnel  should  be  asked  to  verify  the  validity  (or 
limitations)  of  data  as  applied  to  system  performance  estimation,  includ¬ 
ing  methods  of  data  reduction  and  extrapolation  to  full  scale.  The  con¬ 
tractor  should  provide  the  basic  performance  analysis.  The  SPO  should 
make  an  independent  check.  " 

"Cold,  hard  facts  derived  from  cold,  impassionate,  unbiased  testing, 
reported  formally  to  the  SPO.  No  tampering  with  report  by  SPO;  no 
arguing  or  modifying  results  as  a  consequence  of  developer  problems. 
Neither  contractors  nor  SPO's  should  grade  their  own  papers.  Not  to 
say  that  test  agencies  should  shower  the  world  with  test  results.  SPO 
acts  as  a  customer  to  the  test  agency.  The  purchase  he  makes  is  un¬ 
biased  testing — nothing  less.  " 

"I  strongly  support  the  need  for  objectivity  which  I  believe  cannot  be 
rendered  by  either  the  contractor  or  SPO,  both  of  whom  will  and  should 
lean  in  favor  of  the  program.  DoD  test  facility  personnel  should  main¬ 
tain  a  capability  to  analyze  and  report  on  experimental  data  independent 
of  the  SPO  and  contractor.  " 
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EXHIBIT  3— Continued 


"DoD  test  facility  personnel  should  present  the  test  data  and  analyze,  if 
possible.  The  contractor  should  present  his  reaction  to  the  data  as  pre- 
sented--explaining  major  points  of  difference,  if  any.  The  SPD  should 
state  program  impact  and  make  objective  recommendation  to  the  Commander 
AFSC  and  Chief  of  Staff.  " 

"Since  both  the  contractor  and  the  SPO  have  vested  interest  in  keeping 
the  program  going,  a  separate  report  by  independent  test  agencies  should 
provide  top  management  with  their  results  and  analyses  of  both  model  and 
full  scale  testing.  " 

"This  is  a  key  point — one  that  is  now  missing  in  our  system.  In  our  older 
practice,  when  I  got  my  back  up  in  the  laboratory  over  a  technical  point, 
the  SPO  had  difficulty  in  overriding.  This  checks  and  balances  system 
is  absent  now.  The  contractor  wants  to  sell  and  hope  for  the  best — so 
does  the  SPO — and  then  perhaps  trouble.  " 

"The  primary  role  at  major  decision  points  should  lie  with  the  contractor 
and  SPO,  supported  by  their  respective  technical  groups.  The  AFSC  Lab¬ 
oratories  should  be  deeply  involved  in  technical  evaluation  of  the  data. 
DoD  test  facility  personnel  should  play  a  minor  part — primarily  in  inter¬ 
preting  test  procedures  that  may  come  under  question.  " 

"When  it  comes  to  evaluating  the  contractual  compliance  with  predicted 
performance,  matrix  management  might  be  applied  effectively  by  the  SPO. 
The  contractor  and  the  SPO,  backed  by  technical  specialists  from  DoD 
Laboratories  and  Test  Centers,  would  agree  beforehand  on  the  parameters 
to  be  measured  and  acceptable  deviations  based  on  the  experimental  er¬ 
rors  and  past  ground/flight  test  correlations.  The  calibrations  of  the 
facility  and  model  instrumentation,  as  well  as  ground  vs.  flight  simili¬ 
tude,  would  be  essential  factors  to  the  specialists  in  the  data  evaluation. 
The  SPO  has  the  responsibility  for  decisions;  however,  he  could  do  with 
better  advice  in  the  decision  making  process.  " 

"Laboratory/test  facility  personnel  should  carry  equal  weight  in  judging 
performance  of  a  vehicle  as  do  the  contractor  and  SPO.  Independent 
evaluation  reports  are  needed — not  just  test  reports.  " 

"There  have  been  indications  that  the  SPO  and  Contractor  will  look  at  the 
optimistic  side  of  a  set  of  data.  This  is  a  normal  situation  and  unbiased 
engineers  ought  to  be  consulted.  A  procedure  that  should  be  acceptable 
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is  to  have  DoD  test  facility  personnel  analyze  and  report  the  data.  The 
solution  to  this  problem  is  not  always  clear.  However,  in  most  instances 
DoD  test  facility  personnel  have  the  benefit  of  other  programs  and  test 
data  that  is  not  familiar  to  the  contractor  or  SPO.  In  this,  the  interpre¬ 
tation  of  the  data  is  the  major  issue.  " 

"Both  contractor  and  SPO  must  analyze  the  data  in  context  with  other  data 
in  order  to  independently  assess  the  probable  performance.  Test  facility 
personnel  are  rarely,  if  ever,  in  a  position  to  associate  isolated  test 
results  with  a  total  performance  estimate.  Often  the  model  is  not  of  the 
latest  configuration  and  they  are  not  really  knowledgeable  of  the  perform¬ 
ance  trade-offs  that  may  be  acceptable  to  the  SPO. '' 

"The  engineering  offices  in  our  major  SPO's  are  now  quite  capable  of 
making  independent  analyses  of  the  more  significant  wind  tunnel  data  to 
check  on  the  contractor's  conclusions.  This  type  of  in-house  capability 
should  be  continually  improved  enlisting  the  help  of  technical  specialists 
from  the  facilities.  In  particular,  test  facility  engineers  should  provide 
all  the  knowledge  required  to  interpret  the  test  data,  provide  wind  tunnel 
and  balance  corrections,  judge  the  accuracy  of  the  final  results,  and  de¬ 
termine  whether  additional  testing  is  required.  " 

"Require  the  SPO  to  have  two  separate  evaluations  of  significant  go-no  go 
events;  i.  e. ,  one  inhouse  SPO  and  another  by  a  group  of  external  experts. 
The  SPO  should  be  required  to  present  the  results  of  the  outside  experts 
along  with  his  established  position.  " 

"The  configuration  evaluated  will  not  be  the  one  put  into  production;  there¬ 
fore,  attaching  too  much  significance  to  the  wind  tunnel  data  during  the 
evaluation  is  not  proper.  As  far  as  evaluating  the  data,  all  government 
personnel  who  have  the  necessary  skills  should  participate.  " 

"I  would  recommend  test  facility  validation  of  tests  and  evaluation/inter- 
pretation  by  an  agency  independent  of,  or  at  least  in  parallel  with  the 
SPO  and  contractor.  " 

"The  SPO  should  place  more  weight  on  the  advice  of  the  Air  Force  engineer¬ 
ing  community — less  on  contractor's  opinion.  " 

"Data  should  be  obtained  at  DoD  or  NASA  facility  by  personnel  capable  of 
analyzing  the  results.  Competitors  should  test  in  the  same  facility  at 
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approximately  the  same  time  period.  Each  contractor  should  report  the 
results  as  he  sees  them.  DoD  and/or  NASA  personnel,  in  conjunction 
with  the  SPO,  should  compare  the  contractor's  results.  " 

"The  test  facility  people  are  there  to  perform  tests  as  the  contractor  and/ 
or  SPO  may  decide.  The  facility  people  are  not  deciders  of  anything — 
except  how  to  run  the  tests  competently  and  expeditiously,  and  to  report 
exactly  and  completely  what  they  did  and  what  they  found — also  what,  in 
view  of  their  experiences,  the  tests  mean.  " 

"Of  all  the  parties  involved  in  the  interpretations  of  wind  tunnel  data,  the 
contractor  has  most  to  lose  if  the  data  is  not  correctly  representative  of 
the  final  product.  He  should  therefore  have  the  sole  responsibility  for 
analysis  and  application  of  the  wind  tunnel  data.  The  DoD  test  facility 
personnel  should  assume  the  role  of  expert  advisers  on  test  facilities 
practices.  " 

"Both  DoD  test  facility  personnel  and  contractor  provide  assessment  of 
data  to  the  SPO.  He  will  then  have  a  range  of  values — conservative  to 
optimistic — upon  which  he  can  base  his  decisions.  " 

"Since  a  degree  of  subjectivity  is  assumed,  it  is  important  to  establish 
a  procedure  by  which  proper  checks  and  balances  are  created,  while  assur¬ 
ing  that  the  program  does  not  become  bogged  down  in  interpretation  type 
arguments.  It  is  proposed  that  a  dual  semi-independent  role  be  adopted 
by  the  contractor  and  DoD  test  facility.  The  contractor  will  submit  analy¬ 
sis  and  reports  (used  in  establishing  compliance)  for  review  (not  approval) 
by  the  DoD  facility.  Both  parties  thereafter  submit  their  conclusions  and/ 
or  differences  to  the  SPO  as  final  arbitrator.  " 
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REGRESSION  PROGRAM 

Statistical  Approach 

The  aircraft  used  in  this  study  must  be  considered  "random 
samples"  for  statistical  analysis.  A  random  sample  must  meet  the 
criteria  that  it  be  selected  from  a  universe  where  every  member  has  an 
equal  chance  of  being  drawn;  further,  any  method  that  associates  the 
selection  of  an  item  with  the  classification  of  the  item  being  selected 
must  be  avoided.  As  noted  in  the  text  of  Chapter  VI  of  this  report, 
selection  criteria  and  constraints  remove  some  of  the  randomness  from 
the  data;  however,  because  the  sample  is  such  a  large  portion  of  the 
universe,  this  is  not  expected  to  influence  the  results. 

The  dependent  variable  (optimal  test  hours  at  first  flight)  is  almost 
always  affected  by  many  variables,  some  of  which  cannot  (or  the  effects 
cannot)  be  reasonably  quantified;  they  may  sometimes  be  too  numerous 
to  consider.  The  purpose  in  this  regression  is  to  pick  out  a  few  impor¬ 
tant  influencing  variables  that  can  be  identified  early  in  an  aircraft 
development  cycle.  These  variables  are  studied  for  their  possible  aid 
in  illuminating  the  dependent  variable.  Any  good  book  on  statistics  or 
quality  control  will  provide  details  on  the  multiple  regression  analysis 
and  the  tests  of  the  results  (23  and  24).  The  computer  program  used  in 
this  analysis  is  available  at  the  Vanderbilt  Computer  Center  (25). 

The  multiple  correlation  coefficient  (R)  measures  the  degree  of 
association  between  the  dependent  variable  and  the  function  of  the 
independent  variables  as  represented  by  the  regression  equation.  Let 
the  regression  equation  be  represented  by; 


173 


174 


Yc  =  F(X,,  X^.  Xj-X^) 
where: 

Y  -  test  hours  calculated  from  regression  equation 
c 

X  =  independent  variable  (e.  g. ,  thrust,  weight,  etc. ) 
k  =  number  of  independent  variables  in  the  regression 


Other  definitions  which  will  be  needed  are: 

Y  =  dependent  variable  measured  value  from  aircraft  program 

Y  =  mean  value  =  _!.  y 

^  i=l 

n  =  number  of  individual  data  points  (aircraft) . 

/  n  2  ' 

I  ^  ' 

A 

S  is  the  standard  error  of  estimate  of  Y,  and  is  the  variance  of  the 
c 

deviations  from  the  universe  plane  of  regression. 

The  multiple  correlation  coefficient  is  then  defined  as 
2  2 

R  =(1  -  S  V  S  ) 

c 


where: 

"  i=l 
2  2 

The  ratio  S  /  S  is  the  percentage  of  the  variance  of  Y  that  is  unex- 
c 

2 

plained  by  the  regression  equation;  therefore,  R  measures  the  percentage 
of  explained  variance. 

Multiple  regression  equations  usually  provide  more  precise  esti¬ 


mates  than  simple  regression  equations.  However,  on  occasion  the 
addition  of  a  variable  does  not  reduce  the  residual  sum  of  squares  suffic- 
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iontly  to  offset  the  loss  of  a  degree  of  freedom.  It  is  sometimes  desir¬ 
able  to  determine  if  the  addition  of  variables  significantly  improves 
the  ability  of  the  regression  equation  to  predict  the  dependent  variable, 
or  if  the  indicated  improvement  is  due  to  "chance.  "  Also,  one  would 
like  to  be  able  to  compare  the  significance  of  two  regression  equations 
with  different  numbers  of  independent  variables.  If  it  is  assumed  that 
both  universes  are  normal  and  the  samples  are  independent,  this  can  be 
determined  from  an  F  distribution.  Let  the  two  universes  be  designated 
by  subscripts  a  and  b.  It  is  hypothesized  that  the  two  variances  are 

equal:  i.e.,  S  =  S,  .  The  F  distribution  is  the  sampling  distribution 
a  b 

^  2  ^  2  2  *^2  2 
of  the  statistic  S  S,  /  S,  S  ,  where  S  =  nS  /  n  -  k.  The 
abba 

percentage  points  for  the  F  distribution  are  available  in  tabular  form. 
From  calculation  of  the  F-ratio  and  knowledge  of  the  degrees  of  freedom, 
one  can  find  the  probability  (P)  that  an  F  as  large  or  larger  than  the  one 
obtained  would  occur  by  chance;  i.  e.  ,  the  probability  that  the  hypothe¬ 
sis  would  be  accepted  by  chance.  For  most  cases,  the  hypothesis  is 
rejected  if  the  probability  is  greater  than  about  0.  0  5. 

Summation  of  Selected  Data 

Table  32  contains  the  multiple  correlation  coefficients  for  the  case 
of  33  aircraft  (all  except  the  X-15  and  XC-142)  for  the  various  predictors 
and  mathematical  formulations.  It  is  observed  that  the  combination  of 
raw  predictors  and  raw  optimal  hours  gives  the  best  multiple  correlation 
coefficient  for  every  group  of  predictors.  Weight  and  thrust  gave  the 
highest  F-ratio  (80.4);  all  of  the  predictors  are  within  acceptable 
probability  limits.  Comparison  of  predictors  weight,  thrust,  speed,  and 
year,  both  with  and  without  categories,  gives  an  F-ratio  =  0.  809  and 
P  =  0.  503.  Comparison  of  predictors  weight,  thrust  and  speed,  both 


TABLE  32 


MULTIPLE  CORREUVTION  COEFFICIENTS  FOR  33  AIRCRAFT 


Correlation 

Coefficient  (R) 

Predictors 

Case  1 

Case  2 

Case  3  Case  4 

Weight,  thrust, 

3  categories 

speed,  year. 

.  927 

Note: 

Case  1:  Raw  Optimal 
Test  Hours  from  Raw 

Weight,  thrust. 

speed,  year 

.  919 

Predictors 

Case  2:  Raw  Optimal 

Weight,  speed. 

year 

.  801 

Test  Hours  from  Log 
Predictors 

Thrust,  year 

.  781 

Case  3:  Log  Optimal 
Test  Hours  from  Log 

Year 

.  506 

Predictors 

Weight,  thrust. 

speed 

.918 

.  681 

.  744 

.  734 

Case  4: 

Log  Optimal  Test  Hours 

Weight,  thrust 

.918 

.  673 

.  741 

.  694 

from  Raw  Predictors 

Weight,  speed 

.  755 

.  638 

.  670 

.  700 

Thrust,  speed 

.  843 

.  662 

.  701 

.  726 

Weight 

.  525 

.  448 

.448 

.  432 

Thrust 

.  776 

.  575 

.  600 

.  609 

Speed 

.  610 

.  530 

.  573 

.  604 
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with  and  without  year,  gives  an  F-ratio  -  0.  249  and  P  =  0.  627.  Both 
categories  and  calendar  year  were  omitted  as  predictors  after  the  initial 
computer  runs  because  their  contribution  to  the  prediction  equations 
was  not  significant.  Speed  could  have  been  dropped  for  the  same  rea¬ 
son,  but  was  retained  at  the  author's  option.  Because  of  the  dominance 
of  the  SST  data  over  the  statistical  analyses,  none  of  these  regressions 
is  presented  and  these  data  will  be  explained  no  further. 

The  case  for  20  aircraft  is  presented  in  Table  33.  The  ratio  of 
thrust-to-weight  as  a  predictor  was  also  tested,  but  the  results  were 
not  significant  and  are  not  presented  in  the  Table.  The  addition  of  the 
ratio  to  the  predictors  weight,  thrust  and  speed  gave  an  F-ratio  =  0.078, 
P  =  0.  783.  The  case  of  raw  optimal  test  hours  from  raw  predictors  again 
gave  the  better  multiple  correlation  coefficients,  as  demonstrated  in 
Table  33.  The  F-ratios  and  probabilities  of  the  regression  equations 
for  this  case  are  shown  in  Table  34, 


TABLE  34 


F-RATIO  AND  PROBABILITY  FOR  REGRESSION  EQUATIONS 


Model 

Predictors 

F-Ratio 

Probability 

1 

weight,  thrust,  speed,  year 

10.  30 

0. 0005 

2 

weight,  thrust,  speed 

14.  64 

0.0002 

3 

weight,  thrust 

22.  64 

0.0001 

4 

weight,  speed 

16.  15 

0.0003 

5 

thrust,  speed 

20.  02 

0.0001 

6 

weight 

8.93 

0.0078 

7 

thrust 

20.  84 

0.0004 

8 

speed 

6.  57 

0. 0186 

Again,  calendar  year  of  first  flight  did  not  add  significantly  as  a 
predictor.  A  comparison  of  Models  1  and  2  from  Table  34  gives  an  F-ratio 
=  0.004,  P  =  0.999.  Also,  speed  did  not  contribute  much  to  thrust  and 
weight  as  predictors;  comparison  of  Models  2  and  3  gives  an  F-ratio  = 


TABLE  33 


MULTIPLE  CORRELATION  COEFFICIENTS  FOR  20  AIRCRAFT 

Correlation  Coefficient  (R) 


Predictors 

Case  1 

Case  2 

Case  3 

Case  4 

Weight, 

thrust. 

speed,  year 

.  856 

.  778 

.  686 

.  735  , 

Weight, 

thrust. 

speed 

.  856 

.  742 

.  670 

.  735 

Weight, 

thrust 

.  853 

.  656 

.  572 

.  705 

Weight, 

speed 

.  809 

.  741 

.  666 

.  729 

Thrust, 

speed 

.  838 

.  728 

.  652 

.  735 

W  eight 

.  576 

.  518 

.  440 

.  506 

Thrust 

.  733 

.  610 

.  525 

.  626 

Speed 

.  517 

.  437 

.421 

.  479 

Note:  Case  1:  Raw  Optimal  Test  Hours  from  Raw  Predictors 

Case  2:  Raw  Optimal  Test  Hours  from  Log  Prediotors 
Case  3:  Log  Optimal  Test  Hours  from  Log  Predictors 


Case  4:  Log  Optimal  Test  Hours  from  Raw  Predictors 


179 


0.  356,  P  =  0.  565.  Seemingly,  weight  and  thrust  should  be  the  best 
predictors  (Model  3).  However,  it  is  noted  (26)  that  in  the  universe 
itself,  the  standard  error  of  estimate  of  a  multiple  regression  is  usually 
less  than,  or  at  the  most,  equal  to,  the  standard  errors  of  estimate  of 
regressions  of  lower  order.  In  the  universe,  it  is  usally  possible  to 
make  more  precise  estimates  from  a  multiple  regression  equation  than 
from  regressions  of  lower  order.  A  further  concern  is  the  distribution 
of  the  residuals,  the  magnitude  of  the  constant,  and  the  ability  of  the 
regression  equation  to  predict  outside  its  range  of  data.  The  author 
has  already  stated  his  preference  for  Model  2  (thrust,  weight,  and  speed). 
Since  the  purpose  of  these  regressions  is  to  permit  an  estimate  of  optimal 
test  hours  from  limited  knowledge  of  the  aircraft,  all  the  predicting  equa¬ 
tions  are  given  so  that  some  estimate  can  be  made  with  whatever  infor¬ 
mation  is  available. 

Model  1: 

H  =  3530  +  0.  124  T  +  0.  877  S  -  0.  0194  W  -  21.  5  Y 
Model  2; 

H  =  2100  I-0.118T  +  1.171  S  -  0.0181W 
Model  3: 

H  =  2932  +-  0.  143  T  -  0.0241  W 
Model  4; 

H  =  -235  +  0.  0099  W  +■  4.  952  S 
Model  5: 

H  =  524  +  0.  0437  T  +  3.  572  S 
Model  6: 

H  =  4392  t-  0.  0092  W 
Model  7: 

H  -  3524  +  0.  0479  T 
Model  8: 


H  =  2065  +  4.488  S 


where: 

H  -  optimal  test  hours  at  first  flight. 

T  =  total  maximum  sea-level-static  thrust  for  the  aircraft,  with 
afterburner,  if  installed  (pounds). 

W  =  aircraft  gross  takeoff  weight  (pounds). 

S  -  maximum  speed  at  best  altitude  in  miles  per  hour 
Y  =  calendar  year  of  first  flight  (use  only  last  two  digits). 


GLOSSARY 


This  glossary  is  intended  to  provide  identification  and  quick  ref¬ 
erence  for  abbreviations  and  terminology  used  in  this  report  which  may 

be  unfamiliar  to  some  readers. 

AEDC:  The  Arnold  Engineering  Development  Center  is  an  Air  Force 

test  center;  located  there  are  wind  tunnels,  engine  test  cells, 
and  other  environmental  simulation  equipment  which  is  designed 
to  perform  testing  on  aerospace  systems  and  components. 

AFSC:  The  Air  Force  Systems  Command  is  the  Air  Force  organization 

which  has  the  responsibility  for  advancing  aerospace  technology 
and  producing  new  and  better  aerospace  weapon  systems.  AEDC 
and  ASD  are  parts  of  this  organization,  whose  headquarters  is 
at  Andrews  Air  Force  Base. 

ASD:  The  Aeronautical  Systems  Division  is  a  part  of  the  Air  Force 

Systems  Command  and  is  located  at  Wright- Patterson  Air  Force 
Base,  Ohio.  ASD  contains  the  Systems  Project  Offices  and 
expertise  to  direct  contractors  in  the  development  of  aeronau¬ 
tical  systems. 

Backyard  Test  Facilities: 

These  are  test  facilities  (e.  g.  ,  wind  tunnels)  which  are  located 
at  a  contractor's  plant.  Their  construction  may  be  funded  either 
by  the  contractor  or  by  the  government  in  support  of  development 
of  some  system.  They  are  usually  relatively  small  when  com¬ 
pared  to  AEDC  and  NASA  test  facilities. 

DoD:  The  Department  of  Defense  is  at  the  summit  of  a  command  eche¬ 

lon  which  includes  (in  order)  DoD,  USAF,  AFSC,  and  AEDC. 

Facility  Test  Plan: 

This  is  a  detailed  set  of  specifications  which  designates  quan¬ 
tity  of  testing,  types  of  tests,  models,  and  test  facilities  which 
will  support  development  of  a  given  aerospace  system.  The 
test  plan  is  usually  prepared  by  a  potential  contractor  as  a  part 
of  his  proposal. 
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Flight  Prototypes: 

Pre-production  (but  full-sized)  aircraft  which  fly  under  their 
own  power.  Competitive  flight  prototypes  result  when  more 
than  one  contractor  builds  flight  prototypes  for  competition 
via  flight  demonstrations. 

Model  Fly-Offs: 

Scaled-down  aerodynamic  models  of  more  than  one  contractor's 
proposed  final  design  are  subjected  to  competitive  evaluation 
in  wind  tunnels. 

NASA:  The  National  Aeronautics  and  Space  Administration  possesses  a 

number  of  substantial  aeronautical  test  facilities  to  support  its 
mission  of  aerospace  research  and  technology  advancement. 

The  aeronautical  test  facilities  are  primarily  located  at  the 
Langley,  Ames  and  Lewis  Research  Centers. 

Paper  Studies: 

Analyses  which  are  used  as  evidence  to  support  a  contractor's 
claim  for  potential  system's  performance;  in  contrast  to  demon¬ 
stration  of  performance  with  hardware,  models,  components,  etc. 

Service  Funding: 

Sometimes  called  industrial  funding.  The  service  funded  activity 
(AEDC  test  facilities)  operates  with  a  working  capital  fund,  from 
which  operating  expenses  are  paid,  and  which  is  reimbursed 
through  charges  to  organizations  which  use  the  services  (e.  g.  , 
ASD  and  military  contractors).  This  contrasts  with  a  previous 
funding  system  whereby  AEDC  received  an  annual  operating 
fund  and  provided  testing  at  no  charge  to  Air  Force  users. 

SPD:  The  System  Project  Director  is  a  high  ranking  Air  Force  Officer 

who  directs  the  SPO. 

SPO:  The  System  Project  Office  is  established  within  the  AFSC  to 

develop  a  particular  weapon  system.  The  SPO's  for  aeronautical 
systems  are  located  within  ASD. 

USAF :  The  United  States  Air  Force  Headquarters  is  located  within  the 

Pentagon.  AFSC  develops  aerospace  systems  in  consonance 
with  USAF  plans  and  directions. 
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Some  of  the  major  problems  associated  with  the  use  and  usefulness 
of  aeronautical  test  facilities  (wind  tunnels,  etc.)  in  the  development 
of  aerospace  systems  are  defined  and  analyzed.  Contributions  to  the 
study  have  come  from  117  of  this  country's  more  experienced  and  prominent 
aerospace  experts  from  government  and  industry.  The  origin  of  the  facil¬ 
ity  test  plan  and  the  use  of  the  test  facility  to  support  DoD  system's 
development  philosophy  are  explored  and  suggestions  are  made  to  reduce 
conflicting  incentives  and  permit  an  expanded  role  for  the  test  facility. 
The  deficiencies  in  test  facilities  are  shown  to  produce  consequences 
which  are  resulting  in  higher  system  cost  and  less  system  performance. 

The  major  facility  inadequacies  are  enumerated  and  specific  examples  are 
noted  wherein  lack  of  test  capability  has  had  detrimental  effects  on 
system  performance. 

Thirty-five  of  the  recent  aircraft  development  programs  are  studied 
and  evaluated  to  determine  a  procedure  whereby  the  use  of  the  test  facil¬ 
ity  can  be  optimized.  A  multiple  regression  analysis  is  used  to  develop 
a  procedure  for  defining  an  optimal  facility  test  program.  The  procedure 
is  expected  to  be  useful  to  those  who  must  develop  or  evaluate  facility 
test  plans  and  to  those  who  must  predict  test  facility  future  work  loads. 
An  estimating  procedure  for  predicting  testing  costs  is  also  suggested. 

Several  procedural  and  Air  Force  policy  changes  are  recommended. 
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